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ABSTRACT 



A graduate level course was developed and tested 
using the techniques of computer-managed instruction (CMI) to teach 
the techniques of programed instruction. The CMI system included 
diagnostic testing, remediation, data collection, and record keeping. 
A study guide, independent of the computer, directed the student to 
textbooks and other printed materials. A systems approach was used to 
develop the instructional materials for the course. The instructional 
logic included a motivational game in which the student bid points on 
his estimate of how well he could perform on a unit test. Two 
experimental variables were introduced into a study of the 
course — student choice of the unit sequence versus computer presented 
setrsequence and computer-facilitated evaluation of the written 
product versus instructor evaluation. Appendices present materials 
described in the paper, such as behavioral objectives, test items, 
evaluation instruments, bibliography, questionnaires, and program 
procedures. A glossary and reference list are provided. (JY) 
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73. ABSTRACT 

The primary objective of this study was to test the feasibility of develop- 
ing and implementing a graduate, level course by computer-managed instruc- 
tion (CMl). Various approaches to both the management and individualization! 
of instruction were studied. The instructional logic included a motiva- i 
tional game in which the student bid points on his estimate of how well he 
could perform on a unit test. 

The investigation of CMI using student-terminal interaction was motivated 
by the high cost of conventional computer-assisted instruction. This 
approach minimized student terminal time, yet incorporated the best aspects 
of CAI, namely the individualized pursuit of goals, self-pacing and immedi- 
ate feedback, with automatic record-keeping. (continued on next page) 



! 



nn KJRffl 7771 
DD 7 NOV 6S 1473 
S/N 0101-807-681 1 



TFaGE 1, 



Staufuty Clattt^tcatton 

A- 3- 408 



ERIC 



147 






LlW 77 



LINK B 



KEy WORDS 



~R&LE 



WT 



ROLE 



WT 



TTNK C 
WLE 



WT 



FUW 
1 NOD 65 J 
S/N 0101-&07-6S21 



DD 



-1473 



(BACK) 




SeZuZZty ClaZZZ^ZcUZon 

A-31409 



ABSTRACT - continued 



The components of the CMI logic included diagnostic testing, remediation, 
data collection, and record keeping* The CMI system presented no direct 
instruction; a study guide, independent of the computer, directed the 
students to textbooks and other printed materials,, The CMI system gave 
a test after each unit of instruction and provided remedial information 
based on test responses a All components were programmed on an IBM 1500 
Instructional System, 

A systems approach was used to develop instructional material used with 
the CMI system, A conventionally taught course on the systematic design 
of instructional materials was task analyzed, and behavioral objectives 
were written for the specific tasks (units)* Test items consistent with 
the objectives were written and entered into the computer. 

The student received a study guide containing a task description, behavioral 
objectives, and relevant references for each unit. At the computer terminal, 
he received the test for each unit and remedial information if he failed to 
meet criterion,, If a written product was required for a unit, he also 
received the criteria for judging the accuracy of that product. The student 
then chose the next unit to study, bid on that unit, and was signed off. 

The two experimental variables used in the study were student choice of 
the unit sequence versus computer presented set-sequence, and computer- 
facilitated evaluation of the written product versus instructor-evaluation. 
Extensive appendices present examples of the materials described in the 
dissertation, such as behavioral objectives, test items, bibliography, 
evaluation instruments, questionnaires, and BID procedures. 

The course, Techniques of Programmed Instruction, was run by CMI in the 
Fall of 1969 at the Florida State University, with 59 graduate students. 

The students worked at their own pace by scheduling time on the computer 
terminal within predetermined hours. The students tended to seek aid from 
the teaching assistants and fellow students rather than from the course 
professor 0 

The results indicated that the CMI students performed as well as students 
taught the previous year by conventional class -lecture methods. The group 
of students who chose their own sequence tended to perform better than the 
set-sequenced group on the midterm test and final product evaluation. 

They also had higher attitude scores toward CMI and took less time on the 
terminal. However, none of these differences were statistically signifi- 
cant 0 

Developmental costs included acquisition of references and instructional 
materials,, and development and implementation of the CMI logic. This cost 
was $9,297,00* Costs of operating the course, for terminal time and per- 
sonnel, were $3,074.00, which is much lower than the costs of a comparable 
traditionally taught class. 

Future use of the interactive, terminal-oriented CMI system developed in 
this study extends to the development of similar courses which can be 
evaluated in an equivalent manner, as well as to further studies of the 
effectiveness of this course* 
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ABSTRACT 



DEVELOPMENT AND IMPLEMENTATION OF A 
COMPUTER-MANAGED INSTRUCTION SYSTEM 
IN GRADUATE TRAINING 

(Publication No. ) 

Nancy K a Hagerty, Ph 0 D, 

The Florida State University, 1970 

Major Professor; Walter Dick 

The primary objective of this study was to test the feasibility 
of developing and implementing a graduate level course by ccmputer- 
managed instruction (CMI). Various approaches to both the management 
and individualization of instruction were studied n The instructional, 
logic included a motivational game in which the student bid points on 
his estimate of how well he could perform on a unit, test. 

The investigation of CMI using student-terminal interaction 
was motivated by the high cost of conventional computer-assisted 
instruction. This approach minimized student terminal time, yet 
incorporated the best aspects of GAI, namely the individualized pursuit 
of goals, self-pacing and immediate feedback, with automatic record- 
keeping. 

The components of the CMI logic included diagnostic testing, 
remediation, data collection, and r ecordkeeping 0 The CMI system pre- 
sented no direct instruction; a study guide, independent of the computer, 
directed the students to textbooks and other printed materials. The CMI 
system gave a test after each unit of instruction and provided remedial 
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information based on test responses* All components were programmed 
on an IBM 1500 Instructional System* 

A systems approach was used to develop instructional material 
used with the CMI system. A conventionally taught course on the 
systematic design of instructional 'uaterials was task analysed, and 
behavioral objectives were written for the specific tasks (units) . 

Test items consistent with the objectives were written and entered 
into the computer. 

The student received a study guide containing a task description, 
behavioral objectives, and relevant references for each unit* At the 
computer terminal, he received the test for each unit and remedial 
information if he failed to meet criterion. If a written product was 
required for a unit, he also received the criteria for judging the 
accuracy of that product. The student then chose the next unit to 
study, bid on that unit;, and was signed off. The two experimental 
variables used in the study were student choice of the unit sequence 
versus computer presented set-sequence, and computer-facilitated 
evaluation of the written product versus instructor-evaluation* 

Extensive appendices present examples of the materials described in the 
dissertation, such as behavioral objectives, test items, bibliography, 
evaluation instruments, questionnaires, and BID procedures. 

The course. Techniques of Programmed Instruction, was run by 
CMI in the Fall of 1969 at the Florida State University, with 59 
graduate students. The students worked at their own pace by scheduling 
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time on the computer terminal within predetermined hours. The students 
tended to seek aid from the teaching assistants and fellow students 
rather than from the course professor. 

The results indicated that the CMI students performed as well 
as students taught the previous year by conventional class- lectin a 
methods. The group of students who chose their own sequence tended to 
perform better than the set-sequenced group on the midterm test and 
final product evaluation. They also had higher attitude scores toward 
CMI and took less time on the terminal. However, none of these 
differences were statistically significant. 

Developmental costs included acquisition of references and 
instructional materials , and development and implementation of the 
CMI logic. This cost was $9,297.00. Costs of operating the course, 
for terminal time and personnel, were $3,074.00, which is much lower 
than the costs of a comparable traditionally taught class. 

Future use of the interactive, terminal-oriented CMI system 
developed in this study extends to the development of similar courses 
which can be evaluated in an equivalent manner, as well as to 
further studies of the effectiveness of this course. 
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CHAPTER I 



INTRODUCTION 



Technology today is no less praised nor criticized than it was 
in the educational institutions of over 100 years ago (Belanger, 1968). 
Only the focus has changed. Then it was blackboards and world globes 
which came under attack; today it is the computer (Oettinger & Marks , 
1968). Unique to the computer era are two major explosions, one of 
knowledge and the other of population (Bushnell & Allen, 1967). More 
has been learned about science alone in the last 20 years than was 
learned in the previous 20 centuries. Today's students must be able to 
use more information than at any other time in history. At the same 
time, twice as many children stand in need of this education. Yet 
rather than adjust to the need, too many educational systems are con- 
tinuing to teach the same material in the same manner. 

Thorndike (1931) and Dewey (1916) were educational revolution- 
aries in the early part of the present century. Dewey, who was more 
acceptable to educators than Thorndike because of his emphasis on 
education for democracy (Hilgard, 1964), placed great emphasis on the 
individual in the educational process, Educators, while accepting 
Dewey's ideas for individualized education, apparently lacked the means 
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of putting them into operation. Theoretical emphasis was upon the 
individual, yet practice never seemed to adjust for it,. In the early 
1900 's several plans were proposed and implemented (Drake, 1955) which 
allowed children more freedom and individual responsibility for their 
own learning. The Dalton Plan (Parkhurst, 1922) and the Winnetka Plan 
(Washburne, et al., 1926) were two examples. But, here again, the 
educational system was not yet ready to launch individualized instruc- 
tion on a large scale. Then, early in the 1950's, the Air Force 
recognized the need to reorganize military research, development, and 
personnel management to produce a better instructional system (Gagn&, 
1965). The result of this reorganization became what has been termed 
the "systems approach." Educators who have long recognized the impor- 
tance of individualized instruction, saw the potential for using the 
systems approach to education to organize the learning environment in 
the schools. Every aspect of the educational process is considered part 
of a total system geared to individualizing this process and thus 
producing educational outcomes superior to those produced before. 

Problem to be Investigated 

The technology is available to assist teachers and students in 
the learning process. Individualized instruction is a method of allow- 
ing for differences in students and helping each student perform to the 
best of his ability. In addition, the systems approach has been used, 
to develop instructional materials in a more scientific manner. Given 
these factors, one must decide how to use them to better the education- 
al process. Computer-assisted instruction (CAI) has been offered as av 
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solution, but costs seemingly are prohibitive at this time (Mitzel, 
1967) . Computer-managed instruction (CMI) has been proposed as a means 
of using the computer in the educational process more economically, but 
at least four major problems are evident. 

First, there is little consensus as to the definition of CMI. 
The second problem is the question of how one develops and implements 
y materials for a CHI system. A third problem is the redefinition of 

roles of the instructor and the student, caused by the reallocation of 
functions taken over by the computer. And fourth, how feasible is a 
CMI course in terms of hardware and software availability? These four 
problems are investigated in the present study in terms of general 
feasibility of using CMI in a graduate level course of instruction. 

It is necessary to define CMI before showing how the system 



works. Computer-managed instruction can be defined as an. automated 
approach to individualized instruction that performs five functions 
(Hansen, 1970) . It includes diagnostic evaluations and learning pre- 
scriptions e It can counsel students about adaptive learning strategies 
and appropriate career development. It can be used to develop an 
optimal scheduling system to match students with learning resources. 

It car. keep records of instruction and student learning performance. 
While CAI encodes learning materials within the computer system, CMI 
relies upon a rich variety of conventional printed and multi-media 
materials. CMI uses the capability of a computer to manage the progress 
of a student through a program of instruction, testing that progress 
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In the present study, CMI has been defined in terms of the 
components of the system involved, i. e., testing, diagnosis, remedi- 
ation, and recording information- In addition, students benefit from 
the immediate feedback capabilities of a computer, so feedback in con- 
junction with testing (as distinct from instruction) becomes an extra 
function of the CMI system. There was no direct instruction via th* 
computer in the experiment described here. Evaluation items encoded 
within the computer were presented as part of the student-terminal 
interaction, followed by diagnosis and remediation. 

These four components are usually the function of the course 
instructor. In the CMI course delays usually caused by the instructor’s 
time allotments are not evident. Students are immediately informed 
in regards to their status in the course and what they should do next. 

Briggs* has supervised a survey of alternate models for the 
design of instructional materials toward the end of individualizing 
instruction. CMI does not necessarily demand new instructional models, 
but a new approach may be needed in using existing ones. To develop 
the instructional materials for the course, one systems approach was 
implemented as it would have been for any course of instruction. The 
primary difference was the manner in which the materials were used. 

The media were available to the student for individualized, self- 
directed study, while the tests on each learning unit were presented 
via the CMI system. 

Five special questions about scheduling and the recording of 

Unpublished collection of seminar review papers written by students in 
EDR 631, Winter Quarter, 1970, at the Florida State University. 
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student functions were explored in derail in the field test of the 
course 

1- Scheduling: How many terminals would be needed when 

scheduling is determined by the students? 

2. Student Performance: What would be the relative efficiency 

of student control of the learning situation when compared to instruc- 
tor control? 

3. Student Attitudes; What attitudes would students have at 
the end of the CMI course and how would they perceive the role of the 
instructor of the course? 

4. Logistics: In moving from theory to practice, could all 

of the desired components of the CMI system be implemented? 

5. Cost Factors: What would it cost to develop CMI and 

operate the CMI course as compared to conventional instruction? 

To answer the question concerning student control versus 
instructor control of the learning situation, two experimental or 
independent variables were used, the sequence of instruction and the 
evaluation of the materials developed by the student. The 
sequence variable allowed some of the students to choose the sequence 
of course units, while others followed a computer controlled sequence. 
The evaluation variable allowed students to rate cheir own performance 
on instructional materials they had written based on computer pre- 
sented criteria versus instructor ratings based on identical printed 
criteria. 

In summary, problems relating to the development of a computer- 
managed course of instruction were investigated before the actual 
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experiment. The CMI components were defined, as well as the method of 
developing the materials and questions to be answered by the field 
test . 




CHAPTER II 



SURVEY OF THE LITERATURE 



This chapter presents the close relationship between CAI and CMX » 
The former deals with instruction presented on a computer terminal, 
while the latter refers to the management of a course of study via a 
computer -driven terminal. Both use the same equipment, which is 
costly, and both transfer some of the teacher* s traditional roles to 
a machine. Both CAI and CMI offer possibilities and problems never 
encountered before in mass education. Both Ca.n be used in the process 
» of individualizing education. This individualization process is aided 
by use of instructional games. 

Hickey (Ed.) (1968) makes the observation that CAI is an inevi- 
table outcome of the trend toward individualized instruction, of the 
proliferation of information to be acquired, and partly an outcome of 
teacher shortage. On the other hand, Mitzel (1967) states that CAI 
lacks credibility as an instructional tool because of its cost, the 
shortage of good CAI programs, and a lack of detailed plans for con- 
tinuing use of CAI programs. 

Gerard (1967) points out that CAI encourages updating of the 
instructional materials, thus benefiting the teacher, and provides 
more opportunity for teacher-pupil contact. Hansen (1968, p. 173) gives 

7 
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the following aspects of CAI which make it applicable to the individ- 
ualized instruction process: "attentional processes, feedback and 
correctional procedures, retention and transfer and evaluation 
techniques *" 

However, Cooley and Glaser (1969, p. 95) have said, "Before 
any fruitful discussion can begin on how the computer might facilitate 
such education, it is necessary to make explicit just how individuali- 
zation is to be accomplished." They state further that individuali- 
zation is "adapting instructional practices to individual requirements 
(p. 95)*" The adaptation is, in fact, the teacher's recommendation as 
to how to use packaged material for each student. This is termed 
individually prescribed instruction (IPI) . The teacher determines the 
course of instruction and the goals, then offers the student a limited 
choice of materials from which he selects those he feels<*will help him 
reach predetermined goals. The choice given to the student is based 
on the teacher* s estimation of the alternatives which match the 
student's learning profile. IPI is therefore instruction prescribed 
for a given individual, based on his initial capabilities, with less 
concern given to the student's own desires about those goals. 

Esbensen (1968) points out that individualized instruction 
incorporates independent study in varying degrees, depending on the 
ability level of the student and the course of study* Other require- 
ments for individualized instruction are that the student moves at 
his own speed (Esbensen, 1969), that the entering behavior of the 
student match the entry requirements of the course (Gagne, 1967) and 
that the student chooses his own goals. Of the latter, McDaniels 
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(1968, p. 242) comments: "Youth are not too young to choose, only too 

poorly prepared to make choices." 

Sears (1940) studied expectancy levels of students and their 

i- 

ability to make realistic choices using predetermined goals . From 
her study it appeared that when a student was successful, he chose 
realistically, while the same student, if told he was unsuccessful, 
chose erratically. 

Individualization requires organization of the instructional 
materials, according to Bushnell and Allen (1967). Gagne (1968) 
proposed a task analysis based on a learning hierarchy to facilitate 
that organization. He feels that students learn by steps, and if a 
task is accurately analysed a student will not be able to pass the 
objectives for any given step unless he has passed the objectives for 
each preceeding step. Dick (1969) adds to this the use of the systems 
approach in the development of instructional materials. 

The work of Cooley and Glaser (1969) and Esbensen (1968) points 
out the inordinate amount of paper work involved in running an indi- 
vidualized system that met the above requirements. In exploring the 
uses of computers in IPI, Cooley and Glaser (1969) state, "The computer 
cannot be justified if it is simply used to keep records." Even though 
it decreases the need for clerks, it is still more expensive than the 
clerks, they argued; but if one can incorporate other features into 
the use of the computer, then it might be justified. 

Computer-managed instruction can incorporate features other 
than recordkeeping. Current CMI programs are in the research and/or 
development stages. In 1969, Morgan (p. 100) cited the following 
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CMI projects in progress: 

Harry Silberman's work with the Southwest Regional Educational 
Laboratory and the Los Angeles Public Schools; Robert Glaser 
of the University of Pittsburth working with the Oakleaf School 
in Pennsylvania; Donald Torr of Sterling Research Institute, 

Don Tosti of Westinghouse Learning Corporation and Alexander 
Schure of New York Institute of Technology all of whom are 
working with the U.S. Naval Academy. All of these projects 
are sponsored by the U.S. Office of Education. Another large 
project involving Q- ‘s headed by John Flanagan under the 
sponsorship of the American Institute of Research and Westing- 
house Corporation. 

The New York Institute of Technology has operated a system in 
which data processing is all off-line; all evaluation is handled by 

multiple card input. Output records are prepared only upon request 

! 

for specific information. As an integral process, students use CAI 
programs, but the use of CAI course material is not part of the CMI 
system. Data which are generated by the CAI course are fed into the 
CMI system, off-line (O'Dierno, 1968). 

Systems Development Corporation (Coulson, 1968) uses a similar 
CMI system, in that evaluation tests for off-the-shelf materials are 
answered on machine-scorable answer sheets, which are periodically 
optically scanned and scored by the computer. Each day, lists of data 
are supplied to the teacher, informing him of the location of each 
student in relation to his objectives and the status of the class as a 
whole. The system is available to the teacher for interrogation on 
any student as the information is needed. 

Glaser's IPI began without the use of automation (Cooley & 
Glaser, 1969) . The clerical tasks encountered during the first three 
years of operation helped clarify what data were available and 
suggested how they were to be used. This made it easier, according 
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Cooley and Glaser (1969, p. 105) to define their CMI system which 

is called IPl/MIS, the IPI Management and Information System, 

• . The basic data are recorded on optical scan forms by 
clerks located throughout the school* These forms are brought 
together and processed at the IBM 1232 optical scanner. The 
resulting punched cards are then read by the terminal at the 
school and the data edited and added to the current student 
file on disk at the computer. . . 

There are four major functions which the MIS can provide 
in an individualized school: (1) collect data; (2) monitor 
student progress; (3) provide prescription information; 

(4) diagnose student difficulty. 

Flanagan (1970, p. 2) identifies five components of the Project 
PLA.N CMI system as M (l) A set of educational objectives, (2) Learning 
methods and materials, (3) Evaluation, (4) Guidance and individual 
planning, and (5) Teacher development . " Concerning the use of the 
computer in this CMI system, Flanagan (1970, p. 5) states that "most of 
the daily processing involves the transmission of module test cards and 
the return of related messages to the terminal after central processing 
has been completed Further reports are then generated at set times 
during the year and are used to guide the student through his academic 
activities. 

Morgan (1969, p. 100) summarized these CMI systems. 

These studies differ in a variety of ways such as reliance on 
off-the-shelf materials as opposed to developing new instruc- 
tional resources. They also address different academic ievels 
and areas. Their similarities are greater than their differ- 
ences, however. All are designing learning interventions 
based on carefully specified behavioral objectives and all are 
using the computer to mediate between the student, his indi- 
vidual performance on the objectives and the inventory of 
instructional resources related to the objectives. 

The role of the teacher appeared to change or De changed by 

the CMI systems in operation. In Glaser's (Cooley & Glaser, 1969) IPI 



ERIC 



12 



system, introduction of the computer relieved the teacher of some of 
the paperwork and prescribing. In the system at the New York Institute 
of Technology (O'Dierno, 1968) the teacher assumed leqs of a teaching 
role in certain subjects (those taught via the terminal), but remained 
in the traditional role in the others. There is no definite statement 
in any of these studies as to what the role of the teacher was or 
should be in a CMI course. 

Instructional games could be a part of the CMI course, and 
assist the individualized learning process. Nesbitt (1968, p. 1) 
claims that games "seem not only to involve the student and absorb his 
interest but also help him learn better than do other methods." The 
primary intent, according to Boocoek and Schild (1968, p. 14) is "to 
arouse interest and to motivate students to further study." Their 
description of the use of games is concerned with incorporating a game 
into a learning experience or using it in a counseling situation. 
Esbensen (1969) has used BID (see adaptation in Appendix A) as both a 
motivating and monitoring device for the total learning situation. 
Students contract for points, based on their assessment of their own 
abilities, in much the same way as a business man contracts for work 
in the real world. Esbensen (1969, p. 8) states that "the major pur- 
poses of BID are (a) to give each player practice in trying to assess 
correctly his ability to accomplish designated learning tasks, and (b) 
to increase each player's motivation to accomplish the learning tasks." 

In summary, an investigation of CMI systems indicates that 




those in operation utilize the recordkeeping aspects of CAI capabili- 
ties, but fail to take advantage of immediate feedback capabilities or 
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in any way use the computer as an Interaction tool. The increase in 
individualized instruction has emphasized the need for organization of 
educational materials and the development of behavioral objectives. 
Games are used in the educational setting as motivational tools, and 
can transform the entire learning situation into a game context. The 
role of the teacher in an individualized, CMI situation has not been 
defined, but does not appear to be the same as the role of the teacher 
in a traditionally taught class “lecture situation. 
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CHAPTER III 



DEVELOPMENTAL MODELS 



In order to implement a CMI course, it was first necessary to 
develop both the instructional materials and the programming logic for 
the computer system. Dick*s (1969) Model for the Systems Approach to 
the Development of Instructional Materials was chosen as a guide to 
develop the instructional materials, since it was felt that it was 
both concise enough and sufficiently developed to be appropriately 
implemented. The CMI components were defined and then the immediate 
feedback logic for the CMI course was developed. 

Systems Approach 

Dick*s model represents a cybernetic system, as illustrated in 
Figure 1. The steps in that model were used as follows: 

1» The conventional graduate course in programmed instruction 
was chocen for development. 

2. By task analysis, the hierarchial arrangement of skills 
necessary to meet the terminal objective of the course was determined. 

3. Prerequisite skills and knowledge of the students were 
established. 

4. The terminal objective and subordinate tasks were specified 
as behavioral objectives which defined the observable behavior of the 
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Figure 1. Model for the Systems Approach to the Development of Instructional 
Materials (after Dick, 1969) . 
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student and the conditions under which his behavior was to be demon- 
strated. Evaluation items consistent with the behavioral objectives 
were written. 

5 . The instructional sequence which was to be managed by the 
computer was designed. 

60 Available PI texts, journal articles and typed materials, 
with a limited number of materials written specifically for some of 
the objectives, were obtained for each objective in the course. 

7 . Instruction consisted of individualized, self-directed 
reading of reference materials, group discussions, instructor-student 
interaction, and use of instructional games. Students were ramdomly 
assigned to experimental groups as were appropriate to the design of 
the course. 

8. Constant evaluation of both student and system as 
instruction proceeded presented information for analysis of prior 
steps and for improvement of the system as a whole. 

A detailed discussion of the steps of that model* as they were 
used in the development of the course is presented in Chapter IV 0 

Computer-Managed Instruction Logic 

The CMI systems discussed in Chapter II use the same basic 
logic in that all learning is done off the computer; tests and the 
evaluation of their results are performed off-line, and the results are 
available in printouts on a delayed basis. In the present CMI course, 
student-terminal interaction was necessary to provide the student with 



0 



17 



* immediate performance feedback. 

Basically, the logic of the CMI course which was developed was 
as follows: 

1. Identify the learning unit to be studied by the student. 

2. Give the student a test on that unit . 

3* Diagnose areas of weakness according to errors made on 
the test. 

4. Provide the student with remedial information. 

5. Pass him onto the next unit when he reaches criteria. 

In summary, Dick's (1969) systems approach to instructional 

design was used to develop the instructional materials for the course, 
while the CMI system used the components needed to implement a graduate 
level course of study. Those components consisted of on-line testing, 
diagnosis, remediation, and record keeping. The logic provided a 
systematic flow from one unit to the next. 
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CHAPTER IV 



DEVELOPMENTAL PROCEDURES 



The steps presented in Dick's model were used in preparing the 
instructional materials for the CMI course. The documentation of that 
development will be presented in this chapter, as well as the computer 
logic for the CMI system. 

Development of Instructional Materials 
Problem Identification 

The choice of a course for implementation was based on several 
considerations. One, the course should be an ongoing one, providing 
for future use of the materials. Two, it should be one whose content 
is fairly stable. Three, the course must be one which could be 
evaluated using objective questions. Four, it must be one with which 
the author had sufficient knowledge :o develop and implement. The 
course chosen in line with this set of criteria was Techniques of 
Programmed Instruction (identified as EDR 537), taught each fall and 
spring quarter at the Florida State University. 

This course had been taught by the traditional class-lecture 
method, three hours a week. Students were taught the systems approach 
(Dick, 1969) and expected to use it in the development of the product 
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they were required to write for the course, a programmed instruction 
(PI) text. The evaluation of that product took place at the end of 
the quarter. The instructor performed both the product evaluation 
(using a set list of criteria) and final examination evaluation. This 
final examination was an essay type question dealing with uses of 
programmed instruction . 

The course material developed for implementation would consist 
of the same basic information required of previous classes. Greater 
emphasis would be placed on the cognitive aspects (specific knowledge 
required of the student) 3 with periodic tests to evaluate how well the 
students were learning the steps required to write a PI text. Product 
evaluation at each of those steps could assist the student in accurately 
developing his PI text, while the final document evaluation would 
determine the ‘student 1 s grasp of the entire systems approach rationale. 
Both the cognitive and productive evaluations could be managed by the 
computer, while the fina.1 document evaluation would be performed by 
the instructor due to the type of evaluation required. 

Task Analysis 

The task analysis in Figure 2, which was developed for the 

* 

programmed instruction course, outlined 20 units to be completed by 
the student. The tasks, as identified by the analysis, represent the 

skills needed by the student in order to write a PI text. 

•o 

Figure 2 distinguishes between the two types of performance 
required of the student. The units in the task analysis are specifi- 
cally defined as cognitive and/or productive units, indicating 




20 




Figure 2 . Task Analysis of Techniques of Programmed Instruction^ 
showing cognitive and productive units • 
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respectively those units which require learning basic concepts and 
information and those which required use of the concepts to write a 
product. For the first 13 units, some form of learning which could be 
evaluated must take place, whereas Units 14 through 20 required us^ of 
that information. This is represented in Figure 2 as solid lines for 
the cognitive units, broken lines for the productive units, and both 
solid and broken lines when both cognitive and productive events must 
take place. 

Entry Skills and Knowledge 

Students who were to take the course were graduate students 
with the basic abilities implied by that status. It was assumed that 
they would be familiar with the formalities of documentation and would 
be able to document their PI text development. 

In terms of prerequisite knowledge, the students ‘were required 
to have a major interest and background in at least one content area, 
such as statistics, music, or social studies, in order to write the 
PI text required for the cpurse. 

Behavioral Objectives 

The tasks were further defined in terms of one to three 
behavioral objectives which contained the expected behavior of the 
student, under what conditions, and to what level of proficiency. 

While complete mastery is desirable, an 80% criterion wsys decided upon 
to allow for testing errors. 

To illustrate the materials developed for this course, Unit 7, 
which deals with behavioral objectives, has been selected. The unit 
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objectives are presented on the contract sheet in Appendix B. 

Evaluation Items for Behavioral Objectives 

For each cognitive behavioral objective there were approxi- 
mately fifteen questions written of which a student would receive a 
random selection of five. This would permit a new set of questions 
for retests and permit an 80% criterion; the student would receive 
remedial information only it he missed more than one question out of 
five for each objective. A test for a unit contained five, ten or 
fifteen questions, depending on the number of objectives (1, 2, or 3) 
for the unit * 

The test items were either true-false or multiple-choice 
questions. These were not pre-tested for reliability or validity, but 
were presented to three experts for their professional judgment whether 
the items measured the intended objectives. Sample test 'items for 
Unit 7 are presented in Appendix C. 

A pre- and posttest were developed which required students to 
recall information specific to each objective. These were alternate 
forms (see Appendix D and E, respectively) covering the cognitive 
material in Units 1 through 13. The computer tests required students 
to choose correctly from given alternatives, while the pre- and post- 
test (midterm) required short answers and completions. 

For each productive behavioral objective a list of criteria 
was prepared which contained questions relating to the product the 
student was to write. For example, a student working on Unit 7 would 
write behavioral objectives for each of the steps in his task analysis. 
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The criteria questions he would receive for this productive unit are 
presented in Appendix F. These questions helped the student determine 
if he had actually done what was required. 

The product evaluation sheet (see Appendix G) . which was used 
in prior years to evaluate the documentation written by the student on 
his PI text was also used for this f,Mi presentation. No changes were 
made in this evaluation so that comparisons could be made between this 
course and the 1968 class products. 

Instructional Sequence 

Under traditional instruction, the steps in the systems 
approach (Figure 1) were presented in sequence, from one through 
eight. The task analysis included separate units for specific parts 
of each step in the systems approach. This task analysis, based on 
Gagne's (1968) hierarchial learning theory, presumes that a lower unit 
on the task analysis must be learned before any higher unit can be 
learned. Therefore, the units were numbered from the bottom up, with 
the task of Unit 1 being to ’’Identify and describe components of the 
Systems Approach,” and of Unit 20 to ’’Produce a document describing 
Systems Analysis development and standardized evaluation of a Pro- 
grammed Instruction text.” 

Media Selection 

The decision to limit the media aspects of the PI course to 
texts and articles was based on limited time for their investigation 
and/or development. Though there were films, slides, and video tapes 
available, these were not used. 
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References were selected which met the requirements of specific 
behavioral objectives? books, articles, journals, papers and pamphlets. 
In most cases, specific pages from these sources were referenced in a 
student study guide. Texts and reports prepared by t^e students who 
took the course in prior years were also made available. A bibliography 
was distributed which contained all the references for the course. 

Contract Sheets 

When the unit descriptions, behavioral objectives and 
references were completed for each unit, they were incorporated into 
a contract sheet, as presented in Appendix B. This contract sheet was 
the basis for the student's bid. He was to read the objectives for a 
unit and estimate how well he could perform on the unit test, hy so 
doing he was contracting for that unit. He would earn the points if 
he passed, or be penalized if he failed the test. 

Instruction 

The study guide for the PI course contained an introductory 
page, task analysis, flow chart, contract sheets, Unit 20 eva! r, tion 
criteria, a bibliography, and the BID procedures. Only an intro- 
ductory and summary lecture were prepared. Students were free to 
request lectures, small group discussions or any additional help 
needed as the course progressed. 

Instructional Game 

An unsuccessful search for an appropriate instructional game 
began early in June, 1969, and continued into October, 1969- A suit- 



O 

ERIC 



25 



able game would have had to aid in the learning of a unit in the PI 
course, and assist prospective teachers to understand the normal 
instructional situation in the classroom and how it can be realistically 
and positively changed. 

Case studies which had been written initially for another 
game were used. Game proceudres were developed around those case 
studies and the result was COMMENT (Hagerty, 1969), a game which 
used both learning taxonomies and the systems approach to learning. 
Directions were completed, then reviewed by three assistants to 
determine if the game would in fact be instructional. 

This game was available for use by the students during the 
run of the course. 

Evaluation 

Results of the evaluation for this course are presented in 
Chapters VI and VII. 



Development of CMI Logic ’ * V' ■ ‘ i;> - 

The programming logic necessary to implement CMI was developed 
in line with the task analysis and computer systems requirements of 
the IBM 1500 instructional system. Limitations built into this CMI 
course logic are discussed, as well as the type of remediation prer 
seated. Experimental elements and validation checks which were in the 
computer presentation are also described in this section. 

The flow chart in Figure 3 shows the route taken by the 



student through a learning unit. The components chosen for this CMI 
system consisted of testing, diagnosis of weaknesses, remediation, 
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and recordkeeping, but excluded the presentation of instructional 
material. Therefore, the primary flow of a lesson, as indicated 
by Figure 3, was from the unit presentation and bid, to study, test, 
and evaluation of the written product. Remediation was given when 
there was poor test performance on a behavioral objective or when a 
student had failed the productive criteria for the unit. 

The order of presentation for experimental group branches and 
miscellaneous entries in the program is presented in the flow chart 
in Appendix H. 

The CMI logic which was developed could be utilized to 
implement any course which could be similarly evaluated. The limita- 
tions imposed on this program by the author may be overcome without a 
major change in the logic. Present logic permits from 1 to 20 units, 
from 1 to 3 objectives per unit, from one to five random selection of 
questions from an unlimited pool, and unlimited product criteria. Each 
learning unit, consisting of from one to three behavioral objectives, 
was entered as a separate segment on the computer. 

Questions for each unit objective were limited to five, which 
allowed for the 80 percent criterion. Total questions which could be 
entered on the program, from which the five are selected, is limited 
only by disk space (5 million characters) . 

Criteria questions relating to the productive objectives were 
presented in a linear mode. Since the record kept did not identify 
correct or incorrect responses, an unlimited number of criterion 
questions could be entered for each productive objective. 
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When the logic and typing errors for Unit 1 were eliminated 
(debugged), the program coding on that unit was copied (duplicated on 
the computer system) as 19 additional segments. Deletions to adjust 
for less than three behavioral objectives and less than 45 test items 
required further logic adjustments for each unit (segment) . Test items 
were then entered in the appropriate units. There were no test ques- 
tions on Units 14 through 20, therefore the game BID was not used past 
Unit 13. 

An example of a dialogue between a student and the computer on 
Unit 7 is presented in Appendix I. Each unit from 1 to 13 was 
basically the same, while Unit 14 through 20 contained criterion 
questions, but no test questions. 

Remediation 

Responses to the test items for each objective were required 
before the student could proceed. If the response was incorrect, it 
was counted and the correct answer indicated. After all objectives 
for a unit had been completed, the counter was checked. More than one 
error for an objective called up the message, "You should study the 
references for Behavioral Objective (1), H 

Reined i a 1 , assistance was minimal since the intended population 
were graduate students. The references for each unit were presented 
on the contract sheets; the student chose those he wished to study. 

By failing a unit test, the student was made aware that he either had 
chosen references incorrectly or had not studied sufficiently to pass 
the test* He then decided whether to redirect his study or change 
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his study pattern for that unit. 

M iscellaneous Computer "Presentations 

Several additions to the course were provided 1 which were a 

. > 

part of the experimental or control aspects, but not part of the PI 
course itself. The control entries were provided to protect the in- 
tegrity of the data generated. These additions are presented next 
and in Appendix I. 

i 

Since the student was given control of the program presen- 
tation in several places, checks were provided to him which indicated 
if he had signed on to his own program and which unit test he would 
T.eceive. For example, when a student signed on, he was presented, 

,r This is the program for (name) . Have you signed on to your own pro- 
gram? a. Yes b. No." If the student had inadvertently signed on 
to someone else*s program, he chose H b," and was signed off to try 
again. If it was his program, he chose "a," after which he was pre- 
sented, "You will now be given the test on Unit (5) . Is that correct? 
a. Yes b. No." 

If he had studied for the Unit 5 test, he would choose "a 11 and 
proceed to the test. However, if he had studied~Unit 3, or already 
taken Unit 5, he would indicate "b. 11 He would be branched to the 
choose frame where he could choose Units and be presented with a 
display of the units chosen to date by him. If he seriously questioned 
the presentation, he could call the proctor who would adjust the course 
settings. Otherwise, he would continue with the unit test. 
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A "Choose" frame asked* "Do you wish informatiQn on any of 
the following? . . , u This frame allowed the student to choose to see 
any messages which were prepared for the clas3, the names of students 
who had completed the unit on which the student was presently working, 
and a list of units he himself had completed so far. 

Because a study on learner characteristics and , CMI was con- 
ducted simultaneously with the operational phase of this project 
(Gallagher, 1970 ), an anxiety scale (Spielberger , 1966 ) and appro- 
priate branches were entered on each unit. Course logic indicated 
that a student would receive the anxiety scale, test, and productive 
criteria as presented in Table 1. 

TABLE 1 

ORDER IN WHICH THE ANXIETY SCALE, TEST, AND PRODUCTIVE CRITERIA 
WERE GIVEN FOR THREE TYPES OF UNIT ARRANGEMENTS 



Cognitive Unit 


Cognitive and 
Productive Unit 


Productive Unit 


Anxiety Scale 


Anxiety Scale 




Test 


Test 




Anxiety Scale 


Anxiety Scale 


Anxiety Scale 




Productive 

Criteria 


Productive 

Criteria 




Anxiety Scale 


Anxiety Scale 




31 



A section was included on each unit which asked the student, 
prior to taking the test, wnich references were studied and which were 
of the most value and least value in studying for the test. The 

student was instructed to type in the number of selected references as 

f 

they appeared on the contract sheet for that unit. This information 
was used in evaluating the references prepared for each unit. 

Additional information as seemed appropriate was entered for 
each unit. For example, in Unit 4, the student was told that the 
second behavioral objective test questions required a handout. Unit 13 
indicated that BID would not be used for Units 14 through 20, and 
presented the student with his total bid score to date. 

A comment frame was presented after the completion of each 
unit. Students were given 10 lines, 40 characters long, in which they 
could type any statements they wished. The comments were to' be used 
in’ evaluating the course as it progressed, and were used in final 
evaluation of the course. 

Debugging 

After all test items were entered on the computer, each test 
item was examined on the computer terminal for inconsistencies, errors, 
or misunderstandings which could develop because of the display of 
the question. A student experienced in programmed instruction and two 
other graduate students familiar with test construction assisted in 
this process. Revisions were made as necessary. During the first 
two weeks of class, further revisions were made in inproper wording 
and incorrectly coded questions. 
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The first four units were debugged just prior to the time the 
first few students encountered those units. Debugging of the remainder 
of the course preceeded the students by a greater margin. The re- 
searcher and programmer concluded that, upon completion of the de- 
bugging, the program logic was working as it should and the students 
would be able to proceed as planned. 

In summary, the development of the PI course materials was 
facilitated by the use of the systems approach. Development of CMI 
logic using testing, diagnosis, remediation, and recordkeeping 
components was accomplishes, and the PI materials were entered on 
the computer. 
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CHAPTER V 



RESULTS OF DEVELOPMENTAL PROCEDURES 



The development of a course of instruction and the logic for 
CMI presentation of that course was documented, with time and cost 
factors detailed, as well as results of the developmental procedure. 
These will be presented in this chapter. 

The developmental phase covered the dates from March 1, 1969, 
through Noverber 7, 1969. An examination of Table 2 will show how 
many weeks an activity took, as well as the hours actually spent on 
that activity. The task analysis was developed during the months of 
March, April, and May, 1969, and required approximately 40 man-hours 
of work. Upon completion of the task analysis early in June, the be- 
havioral objectives were written along with the test items for the 
cognitive objectives, and the criteria for the productive objectives. 

CMI logic was completed by the middle of July. At that time 
the program was entered on the computer. By the first of August the 
program was working well and the test items for Unit 1 were entered. 
Debugging began in August with Unit 1 and continued through Unit 20, 
which was completed by the last of October. A total of 875 man-hours 
was utilized in preparing the CMI course to the point where it was an 
usable management tool. 
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TIME SCHEDULE OF ACTIVITIES FOR THE DEVELOPMENT OF THE COURSE, TECHNIQUES . OF PROGRAMMED 

INSTRUCTION, USING COMPUTER -MANAGED INSTRUCTION 
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indicates length of time in weeks; 40 indicates number of man-hours spent on activity. 
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Table 3 presents the activities of individuals involved in 
implementing the CMI course. This includes the work of this research- 
er, three assistants, the programmers, and the secretarial staff. 
Actual minute-by-minute records were not kept, but hourly estimates 
were recorded. The course itself covered the dates from September 29, 
1969, through December 8, 1969. Costs per activity are also recorded. 

Development of the materials for the CMI course took approxi- 
mately 280 hours time, while development of the CMI logic in Course- 
writer II language required another 70 hours. At a cost of $4.00 per 
hour, development of materials and logic cost approximately $1,400.00. 
Entering the material on the system took 205 hours, while debugging 
that material took 300 hours. At a cost of $15.48 per hour (computer 
programmer costs at FSU CAI Center, 1968), total computer work cost 
$7,817.40. Secretarial time involved in typing the handout cost 
$80.00. Total developmental costs for this CMI course were approxi- 
mately $9,297.00. 

When the units, test questions and product criterion had been 
entered on the system, there were a total of 20 units, with 22 be- 
havioral objectives and 316 test questions. Fourteen of the units 
required a written product while six did not. This is presented in 
Table 4. 
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TABLE 3 



TIME AND COST UTILIZATION IN DEVELOfttENT AND IMPLEMENTATION OF CKI COURSE 



a Time (in hours) Total Cost ( In dollars) Total 

Activity Researcher Assistants Programmers Secretarial Time Develop Debug Enter Debug ■ Secre- Costs 

Sta££ (Houro)Mater ial Logic Material Material tarial (Dollars) 

($4/hr) ($4/hr) ($15.48/ ($15.48/ ($4/hr) 

hr) hr) 

DEVELOPMENTAL PHASE 



project Design 


40 


-- 


— 


-- 


40 


-- 


160.00 






.. 


160.00 


Task Analysis 


30 


•• 


*- 




30 


120.00 


-- 


— 


— 


— 


120,00 


Behavioral 




•• 


•- 


-- 


— 


— 


— 


— 


— 


•• 


-- 


Objectives 


40 


20 


— 


— 


60 


240.00 


** 


— 


•— 


— 


240.00 


System Logic 


15 


— 


15 


•• 


30 


•• 


120.00 


— 


-- 


— 


120.00 


Test Items 


90 


40 


— 


— 


130 


520.00 


— 


— 


— 




520.00 


References 

Productive 


40 




•** 


5 


45 


160.00 


*• 


•• 




20.00 


180.00 


Criteria 


5 


5 


— 


-- 


10 


40.00 


— 


-- 


— 




40.00 


Contract Sheets 


10 




•• 


10 


25 


40.00 


— 


— 


-- 


60.00 


100.00 


Subtotal 


270 


65 


15 


20 


370 


1.120.00 


280.00 


— 


— 


80.00 


1.480.00 








COMPUTER 


IMPLEMENTATION PHASE 












System Logic 


5 


.. 


40 




45 


... 


... 


696.60 




— — 


696.60 


Test Items 


60 


— 


85 


— 


i45 


— 


— 


2,244,60 


— 


— 


2,244.60 


Debugging 

Productive 


100 


100 


100 


•** 


300 


•• 


•• 


““ 


4,644.00 




4,644.00 


Criteria 


— 


— 


10 


— 


10 


— 


— 


154.80 


— 


— 


154.80 


Anxiety Scale 


mm 


•• 


5 




5 


•• 


“*■ 


77.40 




— 


77.40 


Subtotal 


165 


100 


240 


— 


505 




— 


3,173.40 


4,644.00 


— 


7,017.40 


TvIAL 


435 


165 


255 


20 


875 


1,120.00 


280.00 


3,173.40 


4,644.00 


80.00 9,297.40 














(280 hr) 


(70 hr) (205 hr) 


(300 hr) 


(20 hr) 
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TABLE 4 



NUMBER OF UNITS, COGNITIVE BEHAVIORAL OBJECTIVES, 
TEST QUESTIONS AND PRODUCTIVE BEHAVIORAL 
OBJECTIVES WRITTEN FOR THE CMI COURSE 







Number 


of 




Cognitive 




Productive 


Unit 


Behavioral 


Test 


Behavioral 




Obiective 


Questions 


Obiect ives 


i 


i 


15 


mm r* 




2 


15 






3 


15 




2 


1 


15 


i 


3 


1 


15 


— 




2 


15 




4 


1 


15 


i 




2 


10 




5 


1 


15 


i 




2 


5 




6 


1 


15 


— 




2 


15 




7 


1 


15 


i 




2 


15 




8 


1 


15 


i 




2 


15 




9 


1 


15 


— 


10 


1 


15 


i 


11 


1 


15 


i 




2 


15 




12 


1 


15 


— 


13 


1 


15 


-- 


14 


— 


— 


i 


15 


— 


— 


i 


16 


— 


— 


i 


17 


— 


-- 


i 


18 


— 


— 


i 


19 


— 


— 


i 


20 


-- 


-- 


i 


Total 20 


22 


316 


14 







CHAPTER VI 

METHOD OF IMPLEMENTATION AMD EVALUATION 



In addition to developing a CMI course, this study also 

f 

attempted to answer a number of questions which are related to such 
factors as scheduling, student performance, student attitudes, 
logistics, and costs. The problems investigated are listed below. 

1. Scheduling 

a. How many terminals are needed to accommodate a class? 

b. How do students self-schedule their time during the 
• quarter? 

2. Student Performance 

a. Does student control of the learning situation produce 
results which might be considered similar to instructor control 
of the situation? 

b. How do students sequence the units of instruction? 

c. Which references are read, which are listed as of no 
value, and which as most valuable in studying for the tests? 

d. Do students choose games, groups, or individual study? 

e. How accurately do students evaluate their own work 
when given guidelines? 

f. How accurately do students estimate their own ability 
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39 

to complete a unit as compared with actual performance? 

3. Student Attitudes ^ 

\ 

a. With what attitudes do students leave Jthe CMI course. 

> 

in relation to materials, terminal presentation, and the CMI 
course itself? 

b. How do students perceive the role of the instructor 
in the CMI course? 

c. How often do the students interact with the instructor 
of the CMI course? 

4. Logistics 

a. Were all the necessary components for this type of 
course programmed using the existing 1500 CAI system? 

b. What logistics problems are encountered when running a 
CMI course? 

c. Is the evaluation data desired during the course 
generated when needed? 

5. Cost Factor s 

a. What does it cost to operate a CMI course in terms of 
student time on the terminal and time interacting with the 
instructor of the course? 

b. What is the cost for library materials and handouts 
for the course? 

c* How much does it cost to run a course, in terms of 
cost per student hour of credit? 

d. How does the CMI cost compare with conventional class- 




room instruction? 
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Population 

Subjects for the study were those students who completed the 
course. Techniques of Programmed Instruction, Fall quarter, 1969, at 
the Florida State University. Ten of the students were research 
fellows in a CAI trainee program and 18 were EPDA music trainees. Ten 
students were from the Department of Educational Research, 8 from 
Science Education, 10 from other departments in the College of Edu- 
cation, and 3 from the fields of Biology (1) and Psychology (2) . This 
is presented in Taale 5. 



TABLE 5 

NUMBER OF STUDENTS IN EACH AfcUSA 
WHO COMPLETED THE CMI COURSE 



Area 


N 


Music Trainees 


18 


CAI Trainees 


10 


Educational Research 


10 


College of Education 


10 


Science Education 


8 


Biology, Psychology 


3 


Total 


59 



A total of 55 students completed the course requirements by 
December 8, 1969, and an additional four students completed them by 
January 15, 1970. The students who took the course on an audit basis 
were not included in this study. The 59 students were randomly assigned 
to one of four treatment groups, with the experimental variables in- 
volved being sequencing and product evaluation. Table 6 presents the 
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total number of students in each of the groups, which are described 
below. 



TABLE 6 

NUMBER OF STUDENTS IN EACH TREATMENT GROUP* 





Computer - 

Facilitated 

Evaluation 


Instructor- 

Evaluated 


Total 


Self-regulated 


Sequence 


15 


15 


30 


Set- 


Sequence 


14 


15 


29 


Total 


29 


30 


59 



Design 

Sequencin g 

The sequence of units in the CMI course was either set, i.e., 
the student took the units in a logical order according to the task 
analysis, or was self-selected by the student. Half of the class was 
randomly assigned to the set-sequenced treatment group and half to the 
self-sequenced group. 

Product Evaluation 

As the students proceeded through the course, they chose an 
area of instruction which they system analysed and documented. For 

i- 

those units which required a written product, evaluation was to be done 
either by the student himself, according to criteria, which were pre- 
sented via the terminal, or by the instructor who used criteria 
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identical to that presented on the terminal. Students were allowed to 
revise the product until it met criterion. Students within each of the 
sequencing groups were randomly assigned to either the student- 
evaluated or the ins true tor -evaluated group. 

Students were registered on the computer with appropriate 
switches set for each group (see Appendix H) . Group . 1 students were 
able to choose the units in any order and evaluate their own products 
via a computer dialogue. Students in Group 2 were able to choose the 
unit order, but were to have their products evaluated by the instruc- 
tor according to the identical criteria as they appeared on the com- 
puter for Group 1. Group 3 was presented, in the set-sequence, the 
units from 1 through 20, and they scored their own written products 
according to the criteria on the computer. Those students in Group 4 
were presented the units in the set'-seqUence and had the products 
evaluated by the instructor. 



Equipment 

The FSU-GAI IBM 1500 Instructional System, with IBM 1510 
terminals, was used to manage the course. The terminals consist of 
Cathode-Ray Tubes with keyboard and light pen response capability. All 
responses for the CMI program were keyboard coded, i.e., required a 
typed response from the student. 

Materials 

The materials used in the CMI course were either instructional 
or evaluational. Most of the evaluation instruments were developed 
specifically for the course. 
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The following materials, labeled n EDR 537 Handout, M served as 
a study guide for the students. They were distributed to the students 
at the first class session; 

1. Course Information sheet 

2. Task Analysis of the course content 

3. Flow Chart of the computer logic 

4. Contract sheet for each unit containing: 

a. Unit description 

b. Behavioral objectives for the unit, both cognitive 
and productive 

c * References for tha unit 

5. Evaluation Criteria for the final product 
6- Bibliography (Appendix 0) 

7. BID Rules 

The following reference materials were available for the 
student to use during the course: 

1, Copies of articles and papers 

2, Books 

3, Journals 

4- Instructional Game, COMMENT (Hagerty, 1969) 

Description of Instruments Used in the Study 

The General Information Questionnaire (see Appendix J) was 
designed to elicit biographical data on the student and his prior 
experience with courses and materials related to programmed instruction. 
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A pretest (see Appendix D) and a midterm test (see Appendix E) 
were developed to evaluate the student's ability to recall the cognitive 
information required of the first 13 units. These te^ts were alternate 
forms . 

Student Attitude Toward Computer-Managed Instruction (see 
Appendix K) is an adaptation of Brown's Scale (1966). This scale 
has been used wxth other groups and Brown (1966) reported a 
reliability of .89 for the instrument. 

The Information Questionnaire (see Appendix L) was developed to 
elicit information on attitudes, time allotments and scheduling during 
the course, attitudes of the students about the course, and as a check 
on data generated by the CMI presentation. 

The General Questionnaire Sheet (see Appendix M) was prepared 
as a double check on the area initially chosen by the student and the 
one he used to prepare his PI text, and who he used as a subject matter 
expert in the preparation of the text. The sheet was also used as a 
source for final comments by the students on any aspect of the course. 

The following instruments were administered to all students 
during the first class period (September 29, 1969): 

1. Trait Anxiety Scale 

2. General Information Questionnaire 

3. Pretest for Programmed Instruction Course 

4. Attitude Toward Learning Scale* 

5. OPI Personality Scale^ v 

See Gallagher, 1970, for description and results. 
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6. • State Anxiety Scale* 

The following instruments were administered ty all students 
on the last day of class (December 8, 1969): 

1. Student Attitude Toward Computer-Managed Instruction 

2. Information Questionnaire 

3. Trait Anxiety Scale* 

4. Attitude Toward Learning Scale* ! 

5. Student Assessment of Anxiety Scales 

6. Ranking of difficulty of EDR 537 Units* 

7. General Questionnaire Sheet 

Three folders of instructional materials were prepared for 
each unit which contained all references which were not available in 
books. These folders were available to the student on a loan basis. 

The student could obtain copies of the folder materials at five cents 
per page. Multiple copies of the reference books were purchased for 
inclusion in a library, along with the folders, as well as the evalua- 

t 

tion materials. These were made available for use by the student 
during the course. The library also served as an office for the two 
assistants. 

Procedures 

On September 29, 1969, the first class meeting was held. As 
the students entered the room, they were handed the General Information 
Questionnaire, Trait Anxiety Scale, and the Pretest for Programmed 

*See Gallagher, 1970, for description and results. 
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Instruction Course. When they had completed these, they were given the 
Handout containing the course materials. Each pare of the handout was 
explained an4 the reasons for using Computer-Managed instruction with 
this course were given. The Attitude Toward Learning Scale was then 
given to the students for completion. At the end of the class each 
student was given a personality scale and an anxiety scale, which were 
to be returned prior to beginning work at the computer terminal. 

Students scheduled time on the computer terminal and received 
an introduction to the course during the week following the first class 
meeting. Throughout the remainder of the quarter, pacing was strictly 
up to the needs or abilities of each student. The entire CMI course 
was handled as an educational game according to the rules of BIL (see 
Appendix A) . The units defined by the task analyses made up those 
defined on the contract sheets. BID was included as a motivational 
tool in that it was used to evaluate the student's assessment of his 
own abilities, and points earned were used in partial determination of 
his grade. The students were told that 20 percent of their final grade 
for the course would depend on the BID total, 30 percent on the midterm 
examination grade, and 30 percent on the grade for the final product. 
Also, a student must have completed all of the units in order to receive 
a passing grade for the course. 

A list of areas which instructors felt needed PI cexts was 
prepared and given to the students at the beginning of the course. 

The list served several purposes. It made the student aware of the 
areas in need of programmed texts, and allowed him to choose one in 
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line with his own background and goals. It presented the names of 
professors who were willing to act as subject matter experts in the 
development of the texts. This also meant that they would be willing 
to provide subjects for use in evaluating the written tests. 

There were several selected topics pertinent to the area of 
programmed instruction, but not required as indicated by the task 
analysis, which students could study in order to gain extra BID points. 
For each designated area, such as history of programmed instruction, 
teaching machines, or operant conditioning theory, the student could 
write a one-page review and receive from one to ten extra BID points. 

Activities of the students from September 30 through December 8, 
1969, consisted of those events indicated by the flow chart previously 
presented in Figure 3. These were essentially: sign on to the program, 
take the test, designate the next learning unit, sign off, study and/or 
write, and sign on again. 

The midterm examination was administered to a student when he 
had completed the first 13 cognitive units on the computer terminal. 
Students were free to contact the instructor or the assistants during 
office hours to discuss any problems they may have encountered in the 
course. A record sheet was developed in order for the instructor ano 
the assistants to record who needed help on which topic for how Ion 
A copy of the sheet may be found in Appendix N . 

On December 8, 1969, the class met again, at which time tho; 
students who had not already done so were to turn in their final 
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written product for the course. Four students failed to complete 
course requirements at this time, but were to have them completed 
by January 20, 1970. Students were given a packet of evaluation 
materials to complete • efore they were dismissed. 




CHAPTER VII 



RESULTS OF THE FIELD TEST 

The results of using CMI in presenting a graduate course of 
instruction were analysed under five main headings; (1) Scheduling, 
(2) Student Performance, (3) Student Attitudes, (4) Logistics, and 
(5) Cost Factors, These will be presented in order, with pertinent 
questions under each heading discussed and data presented in response 
to those questions. 



1, Scheduling 

Records were maintained which listed the appointments made by 
the students for terminal time and the mid term examination. Students 
were also asked their preferences for terminal time availability. 

a. How many terminals were needed to accommodate the 
students in the CMI class? 

Scheduling for the CMI sessions on the computer terminals began 
on October 6, 1969. At that time at least five terminals were avail- 
able to the students from 3:30 to 5:00 p. m. on Monday, Tuesday, Wed- 
nesday, and Thursday, and from 7:30 to 10:00 p. m. on Monday nights, a 
total of eight and one-half hours terminal time per terminal, or 42.5 
hours per week. An additional 4.5 hours was provided the 5th and 6th 
week on Thursday night. 
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When asked when they would have preferred to have the terminals 
available, 13 students (22%) would have preferred it in the mornings, 

13 (22%) in the evenings, 11 (187o) all day, and 3 (5%) on weekends . 

Terminal time was scheduled in 15-minute blocks. Use of the 
terminals peaked the week before the midterm examination, as presented 
in Table 7, then drastically declined to the end of the quarter. The 
schedule is only an estimate of use, since students would come to the 
Center without scheduling a terminal and use one if it were available. 
The 627 blocks of total usage shown in Table 7 indicates 156.75 hours 
of terminal time. Actual terminal utilization for the quarter, as 
indicated by the user's file (a computer-maintained time record) was 
259.6 hours. 



TABLE 7 



TERMINAL USAGE BY WEEK FOR CMI COURSE 
AS INDICATED BY SCHEDULE SHEETS 



Week 

in 

Quarter 


Terminals by 15 


-min. blocks 


% of 

Usage 


Available 


Used 


i 


150 


84 


56 


2 


150 


113 


75 


3 


150 


96 


64 


4 


150 


98 


65 


5 


168 


121 


72. * 


6 Midterm 168 


85 


51 


7 


150 


8 


5 


8 


150 


17 


11 


9 


150 


5 


3 


Total 


1380 


627 


45 
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The means, medians, and standard deviations for the four treat- 
ment groups for total terminal time is presented in Table 8. The means 
ranged from 251.53 to 284.28 minutes. The set-sequenced computer- 
evaluated group utilized 32.75 minutes more time on the computer than 
did the self-sequenced instructor-evaluated group. The average time for 
the 59 students was 264.32 minutes, or 4.40 hours per quarter. 

Students needed more time the first seven weeks (see Table 7), which may 
indicate that five terminals available for 52 hours, with most of those 
hours in the first seven weeks 0 could accommodate 59 students a quarter. 
Broken down by week, approximately 8 hours per terminal per week would 
be needed or 1.6 hours per clay for each terminal. 

TABLE 8 

MEAN, STANDARD DEVIATION, AND MEDIAN TERMINAL INTERACTION TIKE ON CMI 
COURSE BY GROUPS AND TOTAL CLASS 
(In Minutes) 



Group 


X 


S.D, 


Median 


Self-Sequenced 

Computer-Evaluated 


260.26 


54.11 


256 


Self -Sequenced 
Instructor-Evaluated 


251.53 


44.32 


250 


Sat-Sequenced 

Computer-Evaluated 


284.28 


46.96 


278 


Set-Sequenced 

Instructor-Evaluated 


262.53 


39.79 


264 


Total Group 


264.32 


46.91 


264 



52 



In analysing terminal time, one student's time was dropped since 
it was three times longer then any other student's. When the time of a 
randomly selected student was compared with that of the above mentioned 
student, he h£d a lower total response time per question and therefore 
the unaccountably high terminal time must have been due to some other 
factor, such as the failure to sign off at the end of a lesson. 

b. How did students schedule their time during the 
quarter? 

The schedules of two students are presented to show the extreme 
variation among individuals in how they spent their time. The first 
student went to the CMI library every day during office hours, read 
nearly every primary reference and had numerous discussions with the 
assistants. She took the tests after studying, setting no special 
deadlines. The second student completed the first 13 units in one week, 
wrote the productive units in the next two weeks, evaluated them and 
completed the documentation well before the other students in the class. 
His schedule is presented in Table 9. 

Both of these students performed in the top percentage of the 
class, and both received A*s. The remainder scheduled themselves be- 
tween these two extremes, with 10 students (17%) working on two units 
per w'aek, 13 (22%) working on three units per week, 12 (20%) fit it 
into their other work, and 9 (15%) had no plan for studying and taking 
the tests. ' There were 8 (13%) who indicated that they were paced by 
the course, but made no further explanation. 
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TABLE 9 

SCHEDULE OF COMPLETION OF UNITS IN CM1 COURSE BY TfiE FASTEST 

STUDENT 

i 



j. 



Date 


Hours 


Unit 

Studied 


Test Passed 


November 


8 


(Saturday) 


2:00 


1-4 




November 


10 


(Monday) 


2:00 


5-7 










: 15 




i 








1:30 




2-7 


November 


11 


(Tuesday 


2:00 


8-10 










: 30 




8-10 


November 


12 


(Wednesday) 


2:00 


11-13 










:30 




11-13 


November 


13 


(Friday) 


3:00 


1-13 










1:30 




Midterm 



Subtotals 


8 :00 


1-13 






2:45 




1-13 




3:00 


1-13 


* 




1:30 




Midterm 


Total 


15:15 







The deadline for the midterm test was November 14. Students 
were to have completed the first 13 units on the computer before taking 
the test. Table 10 indicates when students took the test*- Sis: (10%) 
completed the requirements before November 7, while 15 (25%) took the 
test on November 14c Five percent of the students failed to meet the 
deadline, but presented sufficient justification to allow them to take 
the test the following Monday, The test packet for the midterm averaged 
one hour completion time, with a range from 35 to 95 minutes. 
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TABLE 10 



NUMBER OF STUDENTS TAKING THE MIDTERM TEST 
PRIOR TO AND FOLLOWING THE DEADLINE 
OF NOVEMBER 14, FOR CMI COURSE 



Date 


Number of 
Students 


% 


Cumula- 
tive 7o 


Prior to Nov. 


7 6 


10 


10 


Nov. 7 


8 


14 


24 


Nov. 10 


9 


16 


40 


Nov. 11 


5 


8 


48 1 


Nov. 12 


7 


12 


60 


Nov. 13 


6 


10 


70 


Nov. 14 Deadline 15 


25 


95 


After Nov. 14 


3 


5 


100 


Total 


59 


100 


100 



2. Student Performance 

The CMI and the traditional classes were used to compare product 
evaluation scores between Instructor^ and student-controlled situations. 
The experimental variables, sequencing and evaluation, were used to 
determine their effect on student f s performance on midterm, attitude, 
and product evaluation scores. Use of the PI texts produced by the 
students is also discussed. 

a. Did student control of the learning situation produce 
results which might be considered similar to instructor 
control of the situation? 

To answer the question whether student control of the learning 
situation produced results similar to instructor control, two analyses 
were used* the scores of the final document for each type of class and 
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the differences between three subgroups in the self -sequenced group. 

The scores of the final documents from the traditionally taught class 
(Fall, 1968) were compared with the scores from the CME class. The 
means and standard deviations of the product scores showed a difference 
between means of 1.34 points on a 75 point scale and a difference in 
standard deviations of 4.49 units, as shown in Table 11. There was 
little difference between the groups. 

* 

TABLE 11 

MEANS AND STANDARD DEVIATIONS FOR TWO CLASSES 
ON FINAL PRODUCT SCORES 

Class n X S.D. 

Traditional (1968) 38 50.66 13.21 

CMI (1969) 59 49.32 8.72 



The means and standard deviations for the self-sequenced group 
of 30 students indicated that they received the highest scores on the 
midterm and the final product, had the highest points on the attitude 
questionnaire, and took less time to complete the course than did the 
set-sequenced group, as presented in Table 12. The self-sequenced 
instructor-evaluated group had the highest pretest scores while the 
self-sequenced computer-evaluated group had the lowest pretest scores. 
The possible and obtained range of scores for the five variables are . 
also presented in Table 12. 
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indicates the highest mean for the variable, 
"^indicates the 2nd highest mean for the variable 
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The products written by the students, i.e,., the PI texts, were 
filed each year for use by future classes. Those written by the 
traditional class were not used beyond this, except fo t one which was 

i 

used as a supplement to another course. The texts produced in the CMI 
class were also filed for future use, but students indicated that 55 
(93%) of them would be used in other courses either by the student 
himself or by the professor who served as subject matter expert. 

b. How did students sequence the units of instruction? 

The students in the self -sequenced group chose the course units 
in a variety of sequences as presented in Table 13. Thirteen (22%) of 
the students followed the numerical sequence, 11 (18%) deviated from 
that sequence more than three times, and one student reordered all 13 
units. Six students (10%) reordered three units or less. 

Midterm scores, final product scores and attitude * scores were 
analysed for the self-sequenced groups for; (1) those students who 
changed the sequence three times or less, (2) those who changed it more 
than three times, and (3) those who made no changes in the sequence. 

The means and standard deviations, as presented in Table 14, indicate 
that those who made more than 3 changes in the sequence obtained higher 
scores on the midterm, and had higher attitude scores. Those who made 
no changes had the highest final product scores. 1 
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TABLE 13 

SEQUENCE CHOSEN BY SUBGROUPS IN SELF-SEQUENCED GROUP IN CMI COURSE 

(N = 30) } 















Sequenc 


0 












N 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 










More than 


three 


changes 








‘ 1 


1 


2 


3 


6 


9 


13 


12 


4 


5 


7 


8 


. 10 


11 


1 


1 


2 


3 


6 


7 


5 


4 


8 


9 


10 


11 


' It 


13 


1 


1 


3 


12 


6 


9 


13 


2 


7 


4 


8 


10 


11 


5 


1* 


3 


1 


2 


6 


4 


5 


7 


8 


9 


10 


12 


13 


11 


1 


3 


6 


9 


1 


2 


4 


5 : 


L3 


12 


7 


8 


10 


11 


1 


1 


3 


6 


9 


12 


2 


13 


4 


5 


7 


8 


10 


11 


1 


1 


3 


2 


6 


9 


10 


4 


5 


7 


8 


11 


12 


13 


1 


1 


3 


2 


4 


5 


7 


8 


6 


9 


10 


11 


12 


13 


1 


1 


3 


2 


6 


4 


5 


7 


8 


9 


10 


11 


12 


13 


1 


1 


2 


3 


4 


7 


11 


8 


5 


13 


6 


9 


10 


12 


1 


1 


3 


13 


2 


7 


11 


4 


5 


8 


12 


6 


9 


10 












Three changes or 


' less 








1 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


12 


il- 


13 


1 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


ls 


12 


1 


1 


2 


3 


4 


5 


7 


6 


8 


9 


10 


11 


12 


13 


1 


1 


2 


3 


6 


5 


4 


7 


8 


9 


10 


11 


12 


13 


1 


1 


3 


2 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 


1 


2 


1 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


13 



No changes 



13 


1 


2 


3 


4 


5 6 7 8 


9 10 11 12 13 



& 

This student chose Unit 3 first, then Unit 1, Unit 2, 
Unit 6, and so on. 




TABLE 14 



MEANS AND STANDARD DEVIATIONS FOR THREE SUBGROUPS WITHIN THE SELF-SEQUENCED GROUP 

ON THREE VARIABLES (N = 30; 



Number of 
Group reordered 
units 


N 


Attitude 


Midterm 


Final 


Product 


X 


S.D. 


X 


S.D. 


X 


S.Do 


i 


< 3 


6 


148.50* 


10.25 


96.67* 


4.76 


52.33+ 


5.01 


2 


> 3 


11 


146.91+ 


11.45 


95.81+ 


4.53 


47.54 


9.74 


3 


- none 


13 


141.38 


20.17 


94.77 


5.58 


53.38* 


7.83 



indicates highest score for the variable, 
^indicates 2nd highest score for the variable. 
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c. Which references were read, which were listed as of no 

value and which as of most; value in studying for the tests? 

Prior to each unit test, students identified the references 
they studied as of most value or of no value. This information, to be 
used in revising the resources for the course, is presented in Table 15. 
Groups showed fairly consistant responses in what they read and what 
they found most valuable. Total responses for each unit are presented, 
rather than by group, for ease of interpretation. 

When a student failed a test, his reference selection on second 
attempt was identified as second pass. On this pass the information 
» presented by the student was highly consistent with that of the first 
pass. Percentages under first pass total are for 59 students, while 
those under most value and no value, in Table 15, are for the percent 
of total group who indicated having studied that reference. For 
example, under Unit 1, "DeCecco (1968) (see Appendix 0),"was studied 
by 93% of the group, while of the 93%, 5% felt it was of no value in 
studying for the test and 12% felt it was of most value. On second 
pass, 90% of those students who failed the test studied the reference, 
and of that 90%, 5% felt it wa s of most value. 

On first pass information, references studied by less than 10 
percent of the students were dropped from the analysis. In seven 
cases, students listed references for a unit other than the one for 
which it was recommended. Where students in the 10% group indicated 
whether the reference was of no value or most value, 65%* were in the 
no value category. This category indicated that the references were 
of no value in studying for the test* 
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PERCENTAGE OF 59 STUDENTS WilO STUDIED A REFERENCE, AND PERCENTAGE OF 
THOSE WHO STUDIED IT AND INDICATED IT WAS OF MOST VALUE OR OF NO 
VALUE IN PREPARING FOR THE TEST ON FIRST AND SECOND PASS 



Reference Studied* 


1st: Pas & Percent 


2nd Pass Percent . 




Total 


No 


Most 


Total 


No 


Most 






Value 


Value 




Value 


Value 


UNIT 1 


DeCecco (1968) 


93+ 


5# 


12 


90 


tm 


5 


Taber 3 Glaser & Schaefer (1965 


) 92 


4 


25 


91 


-- 


28 


Alexander 6c Yelon (1969) 


90 


31 


7 


68 


53 


-- 


Hansen 6c Dick (1969) 


95 


16 


67 


100 


8 


71 


Silvern (1968) 


18 


36 


9 


9 


“ “ 


- — 


UNIT 2 


Espich 6c Williams (1969) 


97 


3 


41 


91 


— «• 


54 


Lysaught & Williams (1963) 


93 


4 


71 


83 


~ ** 


.50 


.UNIT 3 


Lysaught 6c Williams (1963) 


92 


5 


42 


89 


_ — . ' 


42 


Glossary (nd) 


93 


-- 


46 


85 


8 


39 


Mathis (1964) 


90 


9 


24 


89 


12 


25 


Morgan 6c Branson (1964) 


13 


“ *"* 


12 


11 


“ **• 


— — 


UNIT 4 


Gagnh 6e Paradise (1961) 


92 


11 


2 3 


100 


12 


17 


Gagn£ (1962) 


92 


4 


36 


100 


12 


12 


Gagni (1968) 


88 


6 


26 


100 


8 


25 


Okey (nd) 


38 


— w 


43 


50 


*• “ 


50 


UNIT 5 


DeCecco (1968) 


98 


— „ 


76 


91 




75 


Fry (1963) 


95 


19 


14 


82 


22 


22 


Lysaught 6c Williams (1963) 


18 


9 


9 


8 


*• "* 


*“ ** 



^Bibliography in Appendix 0. 

+ Percentage of 59 students who studied this reference. 

^Percentage of Total who indicated this reference was of no value in 
studying for the Unit 1 test. 
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TABLE 15 . - "Continued 



Reference Studied 


1st Pass Percent 


2nd Pass Percent 


Total 


No Most 

Value Value 


Total 


No 

Value 


Most 

Value 


UNIT 6 


Espich 6c Williams (1969) 


98 


89 


100 


4 


70 


Lysaught 6c Williams (1963) 


13 


12 


6 






Markle (1964) 


70 


19 2 


59 


19 


13 


UNIT 7 


Mager (1962) 


98 


85 


93 


M M 


78 


Lipe (nd) 


73 


9 11 


21 


“ “ 


~ — 



UNIT 8 



DeCecco (1968) 


85 


2 


61 


100 


_ tm 


67 


Esbensen (1968) 


77 


6 


21 


93 


““ 


27 


Gagni (1965) 


80 


12 


39 


100 


20 


47 



UNIT 9 



Espich 6c Williams (1969) 


98 


_ — 


43 


100 


11 


17 


Glaser (1965) 


78 


23 


19 


72 


8 


39 


Lysaught 6c Williams (1963) 


95 


_ — 


52 


100 


11 


44 



UNIT 10 



Espich 6c Williams (1969) 


97 


_ _ 


47 


100 




44 


Glaser (1965) 


80 


2 


25 


81 




46 


Lysaught 6c Williams (1963) 


95 


2 


44 


100 


— “ 
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UNIT 11 



Thorndike 6c Hagen (1961) 


72 


— 


67 


100 




80 


Burns (1968) 


68 


10 


7 


67 


30 


20 


UNIT 12 


Espich 6c Williams (1969) 


83 


_ _ 


62 


100 


mm mm 


57 


Glaser (1965) 


27 


31 


12 


43 


-- 


9 


Lysaught 6c Williams (1963) 


50 


7 


17 


43 




67 


Tyler 3 Gagn£ 6c Scriven (1967) 


20 


25 


8 


43 


— 


-- 


Rosenoff (nd) 


60 


22 


14 


100 




43 
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TABLE 15 • “ "Continued 



1st Pass Percent 


2nd Pass Percent 


Reference Studied Total No Most 

Value Value 


Total No Most 

Value Value 


UNIT 13 


Fry (1963) 72 16 30 

Briggs (1967) 70 2 35 


89 37 12 

100 33 22 


UNIT 14 


Espich & Williams (1969) 78 -- 74 


— 


UNIT 15 


Espich & Williams (1969) 75 -- 78 

Lysaught St Williams (1963) 58 -- 23 


-- 


UNIT 16 



Espich & Williams (1969) 


77 


— 


76 


«. _ 


— — 




Lysaught & Williams (1963) 


58 




75 


-- 


-- 


~ 


Recommendations ( 1966) 


18 


9 


18 


mm mm 


“ — 





UNIT 17 




Espich & Williams (1969) 


70 




76 




_ 




Lysaught St Williams (1963) 


50 


“ — 


37 


“ *“ 


“ “ 


*" — 


UNIT 18 


Recommendations (1966) 


63 


— 


68 




“ 




UNIT 19 




Recommendations (1966) 


52 




61 




-- 


-- 
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The students ordered a total of 48 copies of articles and papers* 
Of those 48 ^ 37% were for the Glossary of PI Terms, The remainder of 
requests were for references from the entire range of suggested re- 
sources o 

d. Did students choose games , groups or Individual study? 

Students did not sign up for small group discussions, even 
though they were encouraged to do so. However, by talking to them 
it became apparent that they informally met as discussant groups. 

Also, many group discussions began around the CAI terminals and in 
the library as an outgrowth of questions directed to the assistants. 

Indications on the Information Questionnaire showed that 13 
students (22%) wanted optional group discussions scheduled, while 25 
(42%) wanted from one to all classes held, as scheduled by the registrar. 
There were 31 students (52%) who felt that the class should not meet 
at all. In line with this, 37 students (62%) would not have preferred 
"text and lecture classes," 18 (30%) indicated that ,L maybe" they would 
prefer them, and 4 (7%) stated that they preferred text and lecture 
classes absolutely for this course. 

Three students played the game COMMENT . Their response to it 
was favorable and they felt they learned about the use of taxonomies 
and teacher-questioning of students. One student asked if playing the 
game would in any way influence the grade she got for the course. 

When told it would not, she stated that she would like to play it, but 
just did not have the time. Another student could not find two others 
with enough time to play the game. For the most part, the game was 




ignored 
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e. How accurately did the student evaluate his own work when 
given criteria guidelines? 

It was difficult to analyse the data for this question since 
only two students in the computer-facilitated evaluation group failed 
to pass all productive units. On the criterion questions there were 
only three incorrect responses indicated for specific questions, while 
all other responses indicated correct or adequate products. All 
students evaluated by the assistants passed the criteria questions on 
all units. Some students enlisted the aid of the assistants before 
product evaluation in order to have the .product corrected before the 
formal evaluation. 

f. How accurately did students estimate their own ability to 
complete a unit as compared with actual performance? 

Students bid on the first 13 units on their estimate of their 
own performance. Mean bids for the first, second, third, and so forth, 
unit chosen (not necessarily Unit 1, Unit 2, and so forth) are pre- 
sented in Table IS. Mean second bids, after a failure on the first 
bid, were higher on all but two choices. A further breakdown is pre- 
sented in Table 17, which shows bids' of 10 and <10, and pass or fail, on 
first, second and third bid. 

There were 59 students and 13 units, or 767 total possible bids 
for the class. Of that total, 453 of the first bids were for 10 points 
and were passed, and 140 were for 10 points and were failed. These 
figures represent respectively, 77% and 23% of total first 10 point 
bids and 59% and 18% of total first bids. Fifty-nine percent of the 
units on first bid and 78% on second bid were passed. 
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TABLE 16 

MEANS ON FIRST AND SECOND BIDS IN RELATION 
TO ORDER IN WHICH UNITS WERE CHOSEN 
(Maximum Biel ** 10) 





Unit Chosen 


Firsj: Bid 
X 


Second Bid 
X 



1st 


8.79 


8.81 


2nd 


9.52 


9.38 


3rd 


9.14 


9.65 


4th 


8.98 


9.33 


5th 


8.98 


9.73 


6th 


8.88 


9.64 


7th. 


9.39 


10.00 


8 th 


8.41 


9.90 


9th 


8.85 


9.68 


10th 


9.25 


9.53 


Uth 


8.71 


9.73 


12th 


9.41 


9.28 


13th 


8.81 


10.00 



An examination of Table 17 shows that students who bid less 
than 10 points failed more often than students who bid 10, on either 
first or second bid. On all three bids, students tended to bid 10 
points for a unit more often than a lesser amount. 

Students 1 responses to the Information Questionnaire in relation 
to the BID game were categorized and are presented in Table 18, Twelve 
students (20%) stated that BID was a motivational device, while the 
other students made comments which ranged from, TF Made it interesting,” 
to, f lJgh. n A student who consistently bid less than 10 points made 
the remark, f, I do not like the idea of bidding since I always have a 
tendency to be conservative, I would worry about the loss of 10 
points . 11 
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TABLE 17 , 

STUDENT BIDS OF 10 POINTS OR LESS THAN 10 POINTS, PASS OR FAIL STATUS, 
ON FIRST BID, SECOND BID, AND BIDS AFTER TWO FAILURES 



Bid Status 




Group 




Sub- 

Total 


% of 
Sub- 
Total 


% of 
Total 


i 


2 


3 


4 


1st Bid PASS 


118 


116 


105 


114 


453 


77 


59 


of 10 FAIL 


41 


37 


31 


31 


140 


23 


18 


Subtotal 


159 


153 


136 


145 


593 


100 




1st Bid PASS 


17 


25 


23 


24 


89 


51 


12 


of 4.10 FAIL 


19 


17 


23 


26 


85 


49 


11 


Subtotal 


36 


42 


46 


50 


174 


100 




Total 


195 


195 


182 


195 


767 




100 


2nd Bid PASS 


46 


48 


40 


41 


175 


89 


78 


of 10 FAIL 


9 


4 


6 


3 


22 


11 


10 


Subtotal 


55 


52 


46 


44 


197 


100 




2nd Bid PASS 


4 


2 


5 


10 


21 


75 


9 


of <10 FAIL 


1 


-- 


3 


3 


7 


25 


3 


Subtotal 


5 


2 


8 


' 13 


28 


100 




Total 


60 


54 


54 


57 


225 




100 


3rd Bid of 10 


9 


4 


7 


*** 


23 




79 


3rd Bid of <£l0 


1 


— 


2 


3 


6 




21 


Total 


10 


4 


9 


6 


29 




100 
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TABLE 18 

t 

COMMENTS BY CATEGORY ON BID AS USED IN THE' CM! COURSE 



Category Number of Students % 



Fun, enjoyable 
Motivating 
Initially difficult 
Okay 

Not. needed 

Did not like it 

Discouraging 

Ugh I Absurd! etc. 



Total 



7 


12 


12 


20 


7 


12 


10 


17 


6 


10 


7 


12 


6 


10 


4 


7 



59 100 
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3. Student Attitudes 

Data generated by the attitude questionnaire and the information- 
questionnaire x^ere used in answering the following questions concerning 
students' attitudes following completion of the CMI cburse. 

a. With what attitudes did students leave the CMI course, 
in relation to materials, terminal presentation, and 
the CMI course itself? 

At the end of the CMI course, when asked if they would take 
another CMI course, 33 (55%) of the students responded "Gladly," 22 
(37%) "Possibly," and 2 (4%) each for "Reluctantly" and "Never,," This 
is presented in Table 19. 

When asked if they felt that the course material was more 
relevant to their professional goals than that of other courses, as 
presented in Table 20, 24 students (40%) felt it was, 34 (57%) felt 
it was as relevant as others, while 1 (2%) felt it was less so. 

When asked if they felt that the course asked for "too much 
work," 25 (42%) indicated they had "way too much," and 30 (50%) that 
they had r, more than most" courses. This is presented in Table 21. 

Students were asked if the criteria questions for the product 
evaluation were relevant and adequate. The results are presented in 
Table 22. Forty students (67%) indicated that most questions were 
relevant, 8 (14%) that they were extremely relevant, and 9 (15%>) that 
some were relevant. Forty-one (68%) felt that most of the questions 
were adequate, 10 (17%) that some were adequate, and 5 (9%) that they 
were extremely adequate. 
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TABLE 19 

RESPONSES TO "WOULD YOU TAKE ANOTHER COURSE 
i BY COMPUTER “MANAGED INSTRUCTION?" 



Category 


N 


% 


Gladly 


33 


55 


Possibly 


22 


37 


Reluctantly 


2 


4 


Never 


2 


4 


Total 


59 


100, 



TABLE 20 

RESPONSES TO "DO YOU FEEL THE MATERIAL IN 
THIS COURSE WAS RELEVANT TO YOUR 
PROFESSIONAL GOALS?" 



Category 


N 


% 


More relevant than other courses 


24 


40 


As relevant as other courses 


34 


53 


Less relevant 


1 


2 


Not at all relevant to my goals 


— 


— 


Total 


59 


100 
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TABLE 21 

RESPONSES TO "DID YOU FEEL YOU HAD TOO 
MUCH WORK TO DO IN THIS COURSE?" 



Category 


N 


% 


Way too much 


25 


42 


More than most courses, but 
not too much 


30 


50 


Same as most courses 


2 


4 


Less than most courses 


1 


2 


Not enough work 


1 


2 


Total 


59 


100 



TABLE 22 

RELEVANCE AND ADEQUACY OF PRODUCT CRITERIA QUESTIONS 
FOR 59 STUDENTS IN CMI COURSE 



Relevant 



Adequate 



Extremely relevant 


14 


Extremely adequate 


9 


Host were relevant 


67 


Most were adequate 


68 


Some were relevant 


15 


Some were adequate 


17 


Lacked some relevance 


4 


Lacked some adequacy 


2 


Extremely Irrelevant 


0 


Extremely inadequate 


4 



Total 



100 



100 
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Attitudes toward the course were assessed using "Attitude Toward 
Computer-Managed Instruction," a modified form of Brown's Scale (1966). 
The reliability of the scale was 088 for the CMI class. This K-R20 
reliability nearly replicates the .89 found by the author (1966) . Out 
of a total possible score of 200, the class ranged from 95 to 172. The 
summary table of the attitude date is presented in Table 23. Among 
the groups, the set-sequenced instructor-evaluated group had the lowest 
mean (135) and the self-sequenced instructor-evaluated group had the 
highest (146) . 



TABLE 23 

MEAN, STANDARD DEVIATION, AND MEDIAN SCORES ON STUDENT ATTITUDE TOWARD 
COMPUTER MANAGED INSTRUCTION BY GROUPS AND TOTAL CLASS 



Group 


X 


S.D. 


. Median 


Self -Sequenced 
Computer-Evaluated 


143.80 


19.37 


143 


Self-Sequenced 

Instructor-Evaluated 


145.87* 


11.08 


144 


Set-Sequenced- 

Computer-Evaluated 


143.3 6 


13.61 


145 


Set-Sequenced 

Instructor-Evaluated 


135.13 


13.77 


135 


Total Group 

* - - - , 


142.02 


14.99 


144 



High score favorable to CMI. 
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Students were ask. - « to comment on the course, making any 
statements they felt were pertinent. Table 24 presents a represen- 
tative sample of the statements made by the students, and indicates 
in parenthesis the number of students who made similar comments. 

TABLE 24 

RESPONSES TO THE STATEMENT, "WE WANT AN HONEST STATEMENT ON ANYTHING 
YOU FEEL IS PERTINENT TO THE CLASS . " 

Responses 



"I enjoyed taking a course which had been so carefully planned. It 
was rather a shocking blow the first week learning how much work was 
involved but once the logic of what we were asked to do became 
apparent it was appreciated." (9) 

"I felt the course was most relevant. I had never been exposed to 
the 'latest happenings’ in education. I now see many areas of 
application to my own field of interests." (6) 

"This has been an exceptional experience. I wish you could get to 
people in the Biology Department and help them develop such programs. 

1 would like to take other such courses." (2) 

"This course entails enough work to be worth 5 graduate hours.* Other- 
wise Beautiful , recommend it to all graduate students." (11) 

"Much of the reading material was not relevant with the questions 
asked. The reading should be shortened. . . Very good course 0 " (8) 

"The work load was too much for the hours credit. The anxiety tests 
were unclear „ Some test questions were unclear. The references 
should be completely revised, clarified, and programmed." (4) 

"No real gripes about CMI. Better than listening to a prof drone 
on." (2) 

"Great course I The only bad times were when the terminals moved 
slowly due to heavy use. Also, the tasks 14-20 should not all be put 
on the terminal for not computer evaluated people." (4) 




'^Course was offered for 3 credit hours. 
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The general feeling was that the course was helpful, well 
planned, and too much work for the credit received. During the course 
students commented that they were spending too much time on this 
course and not enough on their other courses. When asked why, the 
usual answer was that they enjoyed it more and felt they were learning 
more . 

b. How did students perceive the role of the instructor 
of the CMI course? 

The role of the professor of the course was non-existent as far 
as 35 (58%) of the students were concerned. For 18 students (30%) 
the professor's role was as counselor and guidance person, while 6 
(10%) felt his role was as evaluation expert, and 5 (8%) went to him 
for help as a source expert 0 Table 25 shows the role of the instructor 
as perceived by the students in the CMI course, as well as the per- 
ceived role of the student assistants o Forty-seven students (78%) 
viewed the student assistants as both source experts and guidance and 
counseling experts, while 18 (30%) saw them as evaluation experts. 



TABLE 25 

ROLE OF THE PROFESSOR AND ASSISTANTS 
FOR CMI COURSE 



Interaction Role 


% Interacted with 


Professor 


Assistants 


Source expert 


8 


78 


Counseling and 






Guidance 


30 


78 


Group leader 


0 


10 


Evaluation expert 


10 


30 


Not at all 


58 


2 
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c. How often did the students interact with the instructor 
of the CMI course? 

During the course the professor met with 13 students (22%) in 
his office, none more than once, for from 7 to 30 minutes, for a total 
of 3 hours and 21 minutes. The students went to the assistants and 
fellow students for help much more than they went to the professor, 
as shown in Table 26. 



TABLE 26 

TOTAL INTERACTIONS PER STUDENT WITH THE PROFESSOR, 



ASSISTANTS, 


AND FELLOW 
10 WEEK 


STUDENTS DURING 
COURSE 


THE 






Number of 


Students Who Interacted With 




Professor 


Assistants 


Fellow 


Frequency 






Students 


0 


34 


i 


9 


1 


15 


3 


0 


2 


4 


3 


6 


3 


2 


1 


' 5 


4 


4 


1 


2 


5 




5 


4 


6 


-- 


7 


2 


7 


-- 


2 


-- 


8 


-- 


2 




9-15 


— 


21 


10 


Many 


- — 


13 


21 
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4. Logistics 

Development of the logic for CMI was accomplished in line with 
the requirements of the FSU CAI Instructional system and the components 
desired for the CMI course. Questions relating to the similarity of 
demands of both the requirements and the components are answered next. 

a. Were all the necessary components for this type of course 

(CMI) programmable using the existing 1500 CAI system? 

Implementing the tests on the computer with random selection 
of items presented no special problems, nor did developing the logic 
for providing remedial inf ormation. Diagnosis of weaknesses was done 
in terms of errors per objective for each unit. Recordkeeping con- 
sisted of recording the units chosen or assigned, and counting errors 
made on those units, as well as how many times a student failed a 
unit. Comments and miscellaneous data generation aspects required no 
special adjustments to the system. 

b. What logistics problems were encountered when the 

course was in operation? 

Development of the CMI course logic was complete and accurate 

as far as management of the course was concerned. However, a special 
computer, program (a Coursewriter II function) failed to perform properly 
and 14 students were branched back to the beginning of tests on which 
they were working. 

Several problems in relation to the terminal presentation 
caused some difficulty to the students. 

1. A few correct answers to questions were incorrectly coded 
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as wrong. These were corrected before more than five students had 
proceeded through the units involved. 

2. Students typed in answers, but failed to indicate when they 
were through typing (execute the enter function) until the proctor 
indicated the failure and showed them the correct method. This occurred 
primarily early in the course and had no effect other than to lengthen 
the response time. 

3. Students felt chat the reference questions should be pre- 
sented after the test, as they did not think they knew which references 
were of value unless they had taken the test. The investigator re- 
tained the original sequence since it was felt that a student who 
failed a test would be a better judge on the second pass as to which 
'references were of the most value in studying for the test. 

4. The message frame proved inadequate for the initial problems 
encountered by students, since they required more information than was 
practical for the frame co , andle* A "Special Message Memo' 1 was 
generated for this purpose. 

c. Were the evaluation data desired during the course 

operation generated when needed? 

The investigator requested that the comments typed by the 
students be printed out for evaluation every weekend the course was in 
operation, but due to backlogs, these data were not generated , nor were 
any other data made available for analysis until completion of the 
course hy all of the students. A detailed listing of all responses was 
provided at the end of the course which provided all the information 
desired for evaluation of the program. 
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5. Cost Factors 

The costs incurred in the operation of the CMI course for one 
quarter, with 59 students, was determined and are presented below. 

a. What did it cost to operate the CMI course in terms of time 
on the terminal and time interacting with the instructor? 

Students averaged 4.40 hours terminal time during the quarter. 
Charges per CAI terminal hour at the FSU CAI Center are $3.33. This 
figure is based on 1968 monthly cost of equipment ($17,217.00) ^^.vided 
by utilization per month (250 hours), divided by 31 terminals > plus 
50% for systems preparation, scheduling, and non-utilization. There- 
fore, total cost for 59 students 1 interaction with a terminal for the 
quarter was $864.46. Expenditures for the CMI course are presented 
in Table 27. 

Office hours for the instructors during the course were in excess 
of 480 hours, with 40 hours spent by the professor and at least 440 
by the two assistants. Costs for the assistants, at $4.00 per hour, 
came to $1,760.00. A time-and-topic discussion record was started by 
the assistants (see Appendix N) , but proved impossible to continue. A 
discussion would start by one student asking a question, several other 
students would get involved and it proved difficult to keep track of 
times, names, and topics covered. Thus, individual time records of 
student-assistant interaction did not materialize as a product of the 
study. However, the professor was able to keep a nearly accurate 
record of instructor -student interaction- Actual time interacting 
with students was 3 hours and 21 minutes. Instructor time spent 
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TABLE 27 



DEVELOPMENTAL AND OPERATIONAL EXPENDITURES 


FOR OH COURSE 




Category 


Expenditures 



I* Developmental Costs 



Development of CMI Course Materials and Logic 
CAI Coding and Entering of Materials on 


$ 


1,400.00 


the Computer 




7,817.40 


Secretarial Staff 




80.00 


Total Developmental Costs 


$ 


9,297.40 



II. Operational Costs 
A. Library Costs 





Books 


$ 


* 264.40 




Reproducing Materials 




80.00 




Subtotal 


$ 


344.40 


B 


• Instruction Costs 


A 






Publication of Handout 


$ 


150.00 




Student Assistants 




1,760.00 




Professor (1/10 time) 




450.00 




Subtotal 


$ 


2,360.00 


C 


• Computer Costs 


$ 


864.46 




Total Operational Costs 


$ 


3,568.86 


TOTAL 


PROJECT COSTS 


$ 


12,866.26 
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evaluating the final products was approximately 30 hours. Total 
instructor time spent on the course was determined at one-tenth time, 
or entailed a cost of $450.00. 

b. What did the library materials and handouts for the 
CMI course cost? 

Library costs for books needed by students in the CMI course 
are shown in Table 28. Those books indicated as of no value or which 
were not read are deleted; only those bo'oks indicated as of most value 
in studying for unit tests are included in the last column. 

Costs for reproducing three copies of all primary and secondary 
articles and papers was approximately $80.00. Total library materials 
cost was $344. 40 o Publication expenses for the study guide for the 
course, which was given to the students, was $150.00. 

c. How much did it cost to operate the CMI course in terms 
of cost per student hour of credit? 

Table 28 presents total costs incurred in the development and 
operation of the CMI course. Total operational costs for the CMI 
course were $3,568.86, or $20.16 per student hour of credit for the 
3 credit hour course, with 59 students enrolled. For 5 credit hours, 
which has been approved, the cost per student hour of credit would 
drop to $12.10. 

■ d, How did the CMI costs compare with the conventional 
classroom instri .etional costs? 

Comparative costs of teaching 59 students by the traditional 
method would have been equal to 180 graduate quarter hours, or the 
equivalent of 1.4 professorial positions at $4,500.00 per quarter. 
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TABLE 28 

LIBRARY COSTS FOR BOOKS REQUIRED OF CMI COURSE, AND THOSE BOOKS 
INDICATED AS OF MOST VALUE FOR THE COURSE 





Book 


No. 


Cost/ 

book 


Total 


Cost of 
Books of 
Most Value 


1. 


Espich 6c Williams (1969) 


9 


3.00 


27.00 


27.00 


5. 


DeCecco (1968) 


2 


6.95 


13.90 


13.90 


15. 


Esbensen (1968) 


3 


2.75 


8.25 


8.25 


20. 


Fry (1963) 


1 


6.95 


6.95 


6.95 


23. 


Gagne (1965) 


2 


5.95 


11.90 


11.90 


30. 


Glaser (1965) 


5 


11.50 


67.50 


67.50 


35. 


Green (1962) 


2 


6.95 


13.90 


— 


55. 


Lindvall (1964) 


3 


1.95 


5.85 • 


— 


65. 


Lysaught & Williams (1963) 


3 


4.95 


14.85 


14.85 


70. 


Mager (1962) 


3 


1.75 


5.25 


5.25 


75. 


Markle (1964) 


10 


5,95 


59.50 


59.50 


85. 


Smith & Moore (1966) 


3 


2.95 


8.85 


— 


100. 


Taber, Glaser, 6c 
Schaefer (1965) 


1 


6.95 


6.95 


6.95 


105. 


Thorndike 6c Hagen (1961) 


1 


7.75 


7.75 


7.75 


110. 


Tyler, Gagne, 6c 
Scriven (1967) 


3 




6.00 





78.30 



264.40 
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229.80 
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which equals $6,300.00. The course has been approved as a 5 hour 
course. Therefore the students (59) would produce 295 quarter hours or 
the equivalent of 2.3 professors. At $4,500.00 per quarter, this is 
equal to a cost of $10,350.00 to teach the PI course in the conventional 
class-lecture m de. To these estimates must be added the costs of 
graduate assistants. Thus the CMI course would conservatively cost from 
one-half to one-third the cost of conventional graduate instruction. 

Evaluation Instxuments 

Statistical analyses were performed on the CMI course evaluation 
data to determine the reliability of the midterm and the computer-pre- 
sented questions and the correlation between questions on these two 
instruments. The data on these analyses are presented for informational 
purposes only since the statistical assumptions could not be met in all 
cases, i.e., the normality of the distributions and linear relationship 
between sets of data, due to the use of criterion-referenced tests and 
the resulting highly skewed distributions. 

For purposes of at lysis, test items for each objective were 
assumed to be equal in difficulty and valid in terms of how well they 
measured that objective. The questions were stratified by unit on 
first pass. The first questions presented for Unit 1, for each 
student, were recorded as a pass (1) or fail (0) • The second question 
through the fifteenth question were also recorded. Likewise, the 
first question on Unit 2 was recorded, through the last question on 
Unit 13 o The data thus obtained produced a stratified matrix of 
scores for the total class in terms of items by units and did not 
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represent particular questions. The first question* received was not 
necessarily question 1, for example, due to the random generation of 

i 

them. 

The K-R 20 reliability of the total items, in terms of first 
item received through last item received, by unit, was .777. This is 
presented in Table 29. 

TABLE 29 

ANALYSIS OF VARIANCE SUMMARY TABLE OF RELIABILITY BY 
STRATA FOR COMPUTER PRESENTED QUESTIONS ON FIRST 
PASS, IN ORDER RECEIVED 



Source 


df 


SS 


MS 


Subj ects 


58 


32.99 


.5688 


Strata 


12 


5.38 




It/Strat 


97 


25.20 




S X S 


696 


129.91 ' 


.1866 


Residual 


5626 


712.80 


.1267 


Total 


6489 








Reliability 


Estimate 




R = 1 - ( 


.127/ .569 ) = .777 





Total questions correct for behavioral objective 1, Unit 1, 
were recorded as a single score, then behavioral objective 2, and so 
» forth through behavioral objective 1, Unit 13, for each student. This 
produced a stratified matrix of scores for the total class in terms of 
behavioral objective scores by unit. The stratified K-R 20 re ^i* a bility 
for correct responses per objective was .622, and is presented in 




Table 30 
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TABLE 30 



ANALYSIS OF VARIANCE SUMMARY TABLE OF RELIABILITY BY 
STRATA FOR SCORES ON BEHAVIORAL OBJECTIVES IN 

SEQUENCE 



Source 


df 


ss 


MS 


Subjects 


58 


103.43 


1.7834 


Strata 


12 


26'. 27 




It/Strat 


9 


62.45 




S X S 


696 


570.25 


.8193 


Residual 


522 


352.21 


.6747 


Total 


1297 








Reliability 


Estimate 




R - 1 “ ( 


.675/ 1.783 ) = 0 


622 Strata 


Fixed 



Test questions on the midterm were scored on a range of points 
dependent on the accuracy of the answer by the student • Two judges 
scored the tests, and where discrepancies were found, discussed the 
questions and arrived at a mutually agreeable score* These scores 
were stratified by unit, so that questions 1, 2, and 3 were added to 
obtain a score for Unit 1, question 4 gave a score for Unit 2, and 
so forth. This resulted in a stratified matrix which, when analysed, 
produced a K-R 20 reliability of .593, as presented in Table 31. 

A correlation was run between scores received for each unit 
test and points received on matching midterm questions, by student, to 
determine if there was a relationship between the computer presented 
true-false and miltiple-choice questions and the midterm completion 
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questions. The correlations between seven of the matching items were 
negative, between -.0026 and -.2086. The five positive correlations 
were between .0200 and .2614. These correlations are presented in 
Table 32. 

It must be noted, however, that because the distribution 
of scores for the correlations were not linear, this possibly repre- 
sents deflated correlation coefficients. 



TABLE 31 

ANALYSIS OF VARIANCE SUMMARY TABLE OF RELIABILITY BY 
STRATA ON MIDTERM SCORES 



Source 


df 


SS 


MS 


Subjects 


58 


120.54 


2.0783 


Strata 


11 


1660.57 




It/Strat 


5 


1635.77 




S X s 


638 


913.79 


1.4323 


Residual 


290 


245.40 


.8462 


Total 


1002 








Reliability Estimate 




R = 1 - ( 


.846/ 2.078 ) 


= .593 Strata 


Fixed 



ERIC 



i 



TABLE 32 

CORRELATIONS BETWEEN SCORES RECEIVED ON UNIT TESTS AND ON RELATED MIDTERM QUESTIONS 



Midterm 

Question 1,2,3 


4 


5,6 


7 


8 


9 


10 


11 


12,13 


•14,15 


16 


17 


Unit 

Test 1 


2 


3 


4 


5 


6 


7 


8 


9,10 


11 


12 


13 


r -.0760 


.0416 


.1842 


a 

.0318 


.0026 


.2086 


.0655 


.1299 


.2321 


Cl 

.2614 


.1353 


.0200 


Note; Correlations 


above 


.231 


(P < • 


10) . 
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CHAPTER VIII 



DISCUSSION 



This chapter contains a discussion of the problems investigated 
concerning the implementation and operation of a CMI course of in- 
struction. These are summarized, with a brief statement on unusual 
findings and interpretations of those findings. 

Development 

Development of the CHI materials and logic covered a time period 
of seven months, and involved 350 man-hours of work. Costs for this 
part of the developmental phase came to $1,400.00. The computer im- 
plementation phase required a total of 505 hours of work, for a cost of 
$7,797.00. This CHI course was developed and implemented for less than 
$ 10 , 000 . 00 . 

Operation 

1. Scheduling 

a. How many terminals were needed to accommodate the class? 

Given that a student required from 169 to 387 minutes, or an 
averabe of 4„4 hours terminal interaction time, a CMI course with 59 
students needed five terminals available for 52 hours each during the 

quarter, or approximately 1*6 hours per day for each terminal. 

87 
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b. How did students schedule their time during the quarter? 
Thirty-nine percent of the students had a schedule for com- 
pleting the work during the course, while 61 percent slid not. One 

student completed the first 13 units in one week, taking 15 hours and 

* 

15 minutes for both study time and computer time. 

2, Student Performance 

a. Did students' control of the learning situation produce results 
which might be considered similar to instructor control of the 
situation? 

The final product scores for the conventionally taught class 
(instructor controlled) and for the CMI class (student controlled) show 
highly similar means for the two groups. When the CMI treatment groups 
were compared in terms of degree of student control, ft was found that 
students who were in greater control of their learning schedules per- 
formed slightly better than those students whose sequence was con- 
trolled by the computer. 

b 0 How did students sequence the units of instruction? 

When the self-sequenced group was given the choice of sequence 
for the units, 13 performed the units in the sequence indicated by the 
task analysis and used by the set-sequenced group. The 17 students who 
changed the sequence presented no consistent choice pattern; they 
chose 17 different sequences. 

c. Which references were read, which were listed as of no value, 
and which of most value in studying for the unit tests? 

Those references which students studied were consistently 
listed as primary resources and were also rated most valuable in 
studying for the unit tests. The references listed as of no value 
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can be dropped from the primary reference list and retained as a 
secondary source, since they could provide useful information though 
tangential to those specified in the behavioral objectives* 

d. Did students choose games, groups or individual study? 

Students chose to study on an individual basis, forming in- 
formal discussion groups when needed. The one game developed for use 
in the course was used by only 3 students. The lack of time and lack 

of pertinence to the course were given as the main reasons for not asking 
for group meetings or playing the game* 

e. How accurately did the student evaluate his own work when 
given criteria guidelines, as evidenced by the score he 
received on the final product? 

It was difficult to answer this question since only two 
instances of failure were noted in the computer-evaluated group, and 
none in the instructor-evaluated group* Differences in the final 
product score may be due to inaccurate assessment on the products, 
but this cannot be determined from the data. 

f • How accurately did students estimate their own ability to 
complete a unit as compared with their actual performance? 

Students appeared to be able to predict their performance on a 
unit by using the game, BID, as a tool. Students who lacked confidence 
in themselves about a unit test failed more often than those students 
who had confidence enough to bid 10 points. 

3. Student Attitudes 

a. With what attitudes did students leave the CMI course, in 
relation tc mater ials, terminal presentation and the course 
itself? 

Attitudes in the class toward CHI 'were consistently favorable* 
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The students appreciated the freedom to allocate their learning and 
testing time and felt that the instructor should be available to help 
with problems related to the course o The PI texts that they produced 
were seen as a valuable product and were relevant to current needs of 
of various courses in the university. 

b a How did students perceive the role of the professor and the 
student assistants in this CHI course? 

The role of the professor and the student assistants was pre- 
dominately that of source expert and gu idance-and -counsel ing expert. 

Of secondary importance was the role of group leader or lecturer. 
Evaluation of the products by the students themselves on the computer 
restricted the role of evaluation expert for the professor and assis- 
tants, but retained it for them in relation to the final product 
evaluation. * 

c. How did students interact with the student assistants and 
the professor of the course? 

Students interacted with the assistants and fellow students a 
greater number of times than they did with the professor. Since the 
assistant's role was similar to that of the professor, this does not 
mean that the students did not interact with the instructor of the 
course, since both professor and assistants were instructors. It does 
mean that students sought the help of fellow students to a great 
extent, drawing on their knowledge, and on the aid of the assistants, 
rather than that of the course professor. 
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4. Logistics 

a 0 Were all the necessary components for this type of course 
(CMI) programmable using the existing 1500 CAI system? 

The components, i.e., testing, diagnosis, remedial information 
presentation, and recordkeeping, were all implemented as part of the 
existing IBM 1500 Instructional System at FSU, using the Course- 
writer II programming language. 

b. What logistics problems were encountered when running 
the course? 

The failure of one Coursewriter II function was the only 
logistics problem encountered during the- operation of the course. 

The logic developed for this type of CMI system was complete and 
accurate as planned * 

c. Was the desired evaluation data generated when needed? 

CAI Center backlogs in processing other data analyses requests 
delayed presentation of the data for the CMI course until the end of 
the quarter. Immediate and easily accessible records were not avail- 
able from this CMI system to the instructor. 

5. Cost Factors 

a. What did it cost to operate the CMI course in terms of time 
on the terminal and time interacting with the instructor? 

The cost of operating the CMI course for one quarter, with 59 
students, using one professor and two graduate assistants, came to 
$3,224.46. On the basis of the results of this study, the personnel 
for the course could be limited to one half-time student assistant, 
which would lower personnel expenditures for the course to $350*00 per 
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month, or $875.00 per quarter. Added to the computer costs, total 
operational costs are lowered to $1,739.46, or about half of the 
initial operational expenses * 

b, What did library materials and handouts for the course cost? 

Books purchased for the CMI course cost $264.40. After 

eliminating those references considered of no value, it would cost 
$229.80 to set up a similar library. Handouts cost $150*00 and were 
listed under instruction. This expense could be eliminated by charging 
the student for the manual for the course. 

Cost of reproducing the reference materials, like the cost of 
books, is an initial expense which would not be repeated. 

c. How much did it cost to operate the course in terms of cost 
per student hour of credit? 

The initial operating expenses for the first run of the course 
cost $20*16 per credit hour. A second run would cost $12.10 since the. 
course has been approved as a 5 hour course. When only ongoing costs 
are analysed (personnel and computer time), the cost drops to $5.90 per 
credit hour for a five hour course. 

d* How did CHI cost compare with conventional classroom 
instruction? 

University allocations for the 5 hour course for 59 students 
would be $10,350.00, or $35.10 per credit hour. This is compared to 
the $5.90 per credit hour required to teach the CMI version of the 




course. 
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Evaluation Instruments 

Analysis of the testing data from the CMI course was difficult 
because criterion-referenced materials were used., and the resulting 
distributions were highly skewed. The relationship between tests did 
not present the needo.i normal distributions for statistical analysis. 

The data indicated relatively useful midterm and computer tests. 
The correlations between related questions on the midterm and the 
computer tests, though deflated due to non-linear relationships, pre- 
sent a unique result. On seven units, those students who missed the 
the most questions on the computer appeared to achieve the greatest 
number of points on the midterm, which may indicate that the computer 
presentation was a learning experience for the students. 

In summary, students for the most part were unable to define a 
set schedule for completion of the course units, but studied for them 
in a random fashion. The findings indicate that students in this CHI 
course were able to perform as well as those in a traditional class. 
Those students given more control of the learning situation achieved 
higher scores on the computer tests and final product, and had a 
higher attitude score toward CMI* 

The role of the instructor was viewed predominately as that of 
source expert and guidance counselor, with a secondary role as eval- 
uator t 

The CMI logic appeared to perform as intended, while data on 
student progress aspects were lacking. Materials prepared for t 1 ' 
course appeared to serve the needs of the students who left with 
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relatively favorable attitudes toward computer-managed instruction. 

Cost effectiveness data indicates that presentation of this 
course via CMI greatly reduced the operational cost per credit hour 
of instruction. 

There were a few unusual findings. The self-sequenced computer- 
evaluated group took less time to complete the computer dialogue than 
the other groups. This group evaluated their own products on the 
terminal, which shou i have required mere terminal time. The reason for 
the lower time may be that this group chose the units they were pre- 
pared for and were able to proceed without delay. However , the self- 
sequenced instructor-evaluated group should, for the same reason, have 
had a mean terminal interaction time less than that of the self- 

sequenced computer-evaluated group. This was not the case, however. 

* 

Students expressed a preference for more interaction with the 
course professor. This contrasts with the actual number of interactions 
with the professor. Even though he was available , students did not go 
to see him as much as they indicated they wanted to. 

Students’ bids were fairly consistent throughout the first 
thirteen units. They made little change in predicting their own 
ability to accomplish a unit, possibly because graduate students are 
well aware of their ability to perform and therefore consistent in 
their estimation of success. 

Midterm scores for the group who changed the sequence of the 
units more than three times were higher than for those who were given 
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the opportunity, but did not change the sequence. However, the latter 
group performed better on the final product. At first glance this may 
appear to be inconsistent, but may point to the need for the set 
sequence in relation to productive unit presentation. 




CHAPTER IX 



SUMMARY AND RECOMMENDATIONS 



The use of CMI for a course of study was facilitated by the 
organization of the material for the course in line with the "Systems 
Approach" requirements. Behavioral objectives were set up, indicating 
to the student what was expected of him during the course of instruc- 
tion The use of the computer in the course for monitoring student 
experiences facilitated recordkeeping and data collection;, and gave 
immediate knowledge to students about their position in relation to 
course requirements * Immediate scoring and presentation of necessary 
remedial information was facilitated since the tests were part of the 
computer presentation. 

Costs of operation were lower than those of a traditionally 
taught class. Students using self-sequenced computer-facilitated 
evaluation were able to move through at their own pace, with no loss 
in learning or productivity, and with only minimal interaction with 
the professor. Individualized instruction gave the student the 
opportunity to explore, without the usual confinement of specified 
class-lecture schedules and lock-step course assignments 0 

Computer -managed instruction, in which instruction is by in- 
dividualized pursuit of goals, with testing and management aspects 
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programmed on the computer, facilitated the educational process and 
provided for a self -directed approach to learning* 

Future Research Using This CMI Logic 

The current investigation was undertaken to implement a CMI 
course of study, and to explore learning variables using the course as 
a tool. The logic developed for this CMI could be used with any set 
of instructional materials which have been organized and can be 
evaluated using true-false and multiple-choice questions. Appropriate 
unit descriptions, test questions and criteria would replace the 
existing ones. 

Future users of the course logic could make several changes in 
the CMI system to make it operate more effectively. Separate counters 
could more accurately record the units CQmpleted and be available in a 
useful form on the users file. A quick examination of switch settings 
would indicate those units chosen by the student.. Also, a proctor 
message could indicate to the instructor when a student had completed 
a unit, which objectives were missed, and how many questions per 
objective were missed. This could be used at the end of the day to 
up-date student records. 

Student performance on midterm and final product evaluation, and 
attitudes toward CMI were basically the same for the four treatment 
groups, with slightly higher means and slightly lower terminal time 
registered by the self-sequenced group. In terms of cost effectiveness, 
it would appear that the most efficient method for. a student to proceed 
through a course of this type would be to choose his own sequence and 
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receive the computer -facilitated evaluation for his written product. 

This could mean that the course instructor would fill the role of 
guidance and counceling expert, source expert and lecturer or small 
group discussion leader as needed, and evaluation expert at the con- 
clusion of Unit 20. The professor* s time could be spent at tasks 
other than lecturing at appointed class sessions. 

The evaluation role of the instructor presents a problem for 
the system currently in operation in educational institutions if they 
use this form of CMI . A class of 20 or 30 students would present little 
hardship to the professor in terms of end of the quarter evaluation. 
However, if the class membership numbered in the hundreds, as it well 
could with this type of CMI presentation, the end of the quarter 
evaluation could pose a hardship for the professor. For this reason, 
it would be to the advantage of both professor and student if the 
student could enter and finish the course at any time during the 
year, thus staggering the termination date and end-of -course 
evaluations . 

Since self -sequencing appeared to have some effect on the 
student 1 s performance, it might be that the productive and cognitive 
aspects of this course could be separated as independent programs . 

A student could choose either the cognitive test or the productive 
criteria for any unit at any time, allowing him even greater freedom 
of choice than self-sequencing alone. Also, the course could be 
offered for variable credit. A student interested only in the 
cognitive aspects of the systems approach could take the first 13 units 
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and the midterm for fewer credits than the student who desired both 
the cognitive and productive aspects of the course. It would be 
difficult to produce only the products without the cognitive work, 
unless the student had had prior experience with the systems approach. 

The use of BID with this graduate course did not prove as 
motivational as anticipated. Students felt it was a threat, since 207 o 
of their final grade was to be based on their final BID total. This 
game could possibly be more effective if used strictly as a motiva- 
tional device and unrelated to the course grade. 

With the information gained from the students' evaluation of the 
references;, the resource materials used in the course can be greatly 
reduced, with those retained providing sufficient information for the 
students to meet the objectives. 

The influence of the CMI presentation extended to many colleges 
of the university. Those professors who served as subject matter 
experts planned for future classes to use the PI texts generated by 
the students in this course. They are also aware of CMI and how it 
can be used* 

In summary, a computer-managed sequence of instruction, as 
developed and implemented in this study, can reduce the cost of 
educating graduate students with no loss in learning taking place 
when compared with the traditional class. Future studies can build 
on the findings presented here and possibly even increase the amount 
information retained by the students. 
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BID: A GAME OF SELF-DIRECTED LEARNING 
By Thorwald Esbensen* 



PURPOSE : 



1. To give each player practice in assessing his ability to 

accomplish designated learning tasks, and 

2. To increase each player's motivation to accomplish the 

learning task, 

ROLES: 

1. Teacher 

a. Write performance objectives for the course. 

b. Be available during specific office hours to assist 
student with any problems related to the course. 

C. Provide subjective evaluation criteria for student's, 
written material. 

2. Student 

a. Work on task. 

b. BID on points representing his assessment of his ability 
to accomplish the cask. 

c. Keep records of his own on task BID for passed or failed 
and total points. 

d. Write, in stages, a product which will be the major 
requirement of the course. The product is to be graded 
as each step is completed. No BID points will be used 
in product evaluation. 

3 . Computer 

a. Introduce student to the game BID and the computer. 

b. Present tasks, tests, and evaluate results. 

c. Indicate when remedial work is necessary to pass test. 

d. Keep score of BID points for each student, for Units 1 
through 13 . 



*The version of the game, BID, is an adaptation of the one designed 
by Esbensen (1969). 
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OBJECTS OF PLAY: 

1. Score Sheet. 

2. Contract Sheets. 

3. Bibliography. 

4. Reference material in the Library. 

NATURE OF COMPETITION: 

1. Student competes with himself for points. 

2. Game is non-zero-sum. 

3. The first ten students who successfully complete a unit are 
placed on an EXPERT list, available on the computer for 
reference by the other students. Assistence by the EXPERTS 
is strictly voluntary. 

RULES OF PLAY: 

1. Student contracts for a unit. 

2. Student examines the Contract Sheet for that unit. He; 

a. notes objective (s) of the unit, 

b. checks references. 

3. Student performs the necessary study to prepare himself for 
passing the test on the ob jective ( 0 • 

4. Student returns to the terminal and takes the test. 

5. Student may avail himself of the teacher 1 s time whenever he 
needs assistance. Re may also use other students and/or 
EXPERTS for assistance. 

6. Student writes material at each step which will be used to 
produce the product for the course. This will be graded 
Pass or Fail as each step is completed. 

7. Units 1 through 13 must be comp eted before the student can 
receive a passing grade for the course. 

SCORING PROCEDURE: 

1. Student begins with 20 points. 

2. For each unit he may BID from 1 to 10 points on his assessment 
of his ability to receive a pass on the unit test. 

3. If he passes, he adds the points to his running total; if he 
fails, he subtracts the points. For example: 



Contract 

Number 


Value 

BIDC1-10) 


Test 

Results 


PASS: 

Add 

Score 


FAIL: 

Subtract 

Score 


Total 

Running 

Score 


i 


5 


pass 


+5 




20 


. 25 


2 


7 


fail 




-7 


18 


2 


5 


pass 


+5 




23 


3 


8 


i 
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4. The computer will also keep score, but it is recommended 
that the student keep track for his own reference as to 
where he is and where he is going. 

5. Total Possible Points -- 150. 

a. A total of 207„ of the final grade will depend on 
BID points earned. 

6. Additional BID points may be earned by the student for turning 
in one-page summaries of pertinent topics. 



BID SCORE SHEET 



Contract 

Number 


— 

Value 

BID(l-lO) 


Test 

Results 


PASS: 

Add 

Score 


FAIL: 

Subtract 

Score 


Total 

Running 

Score 


+20 










» 








































































1 





































































































UNITS 1 2 3 4 5 6 7 8 9 10 11 12 13 

Circle number when chosen--{i) through when test is passed--^). 
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CONTRACT SHEET 
UNIT 7 

IDENTIFY BEHAVIORAL OBJECTIVES WHICH HAVE BEEN WRITTEN 
CORRECTLY. WRITE BEHAVIORAL OBJECTIVES FOR YOUR 
AREA OF INSTRUCTION. 

Productive Behavioral Objective 

Student is to write Behavioral Objectives for each unit accord- 
ing to the three required elements for behavioral objectives. 

Cognitive Behavioral Objective 1 

The student will be able to identify correct descriptions, the 
necessary conditions and specific characteristics of a behavioral 
objective . 

Cognitive Behavioral Objective 2 

Given sample objectives, the student will be able to identify 
the components which are missing. * 

References 

Primary 

500. Lipe, J. G. Are instructional objectives essential? pp . 1-11. 

70. Mager, R. F. Preparing instructional objectives . pp . 1-60. 
Secondary 

310. Atkin, J. M. Behavioral objectives in curriculum design, 
pp. 27-30. 

355. Burns, R. W. Behavioral objectives; A selected bibliography, 
pp. 57-58. 

385. Deter line, W. A. The secrets we keep from students. pp. 7-10. 
485. Kapfer, M. B. Behavioral objectives and the gifted. pp . 14-16. 
J>5. Lindvall, C. M. Defining Educational Objectives . pp . 1-77. 
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TEST QUESTIONS 
UNIT 7 



Behavioral Objective 



1 



The student will be able to identify correct descriptions, the 
necessary conditions and specific characteristics of a 
behavioral objective. 

Questions 

070101 One of the major reasons for using behavioral objectives is that 
they allow the teacher to be more efficient in evaluating the 
effect of instruction. 

a. True b. False 

070102 Behavioral objectives describe: 

a. how the learner will undertake a learning sequence. 

b. what a learner will do upon completing a learning sequence. 

c. the methodology that will be utilized by the teacher. 

d. All of these. 

070103 In a well stated behavioral objective, which of the following 
should be identified by name and defined in words of action? 

a. criterion test c. observable response 

b. criterion performance d. important conditions 

070104 Which of the following terms should not be used when writing 
behav iora 1 ob j ec t ives ? 

. a. measure b. list c. contrast d. comprehend 

* 

070105 The terms behavioral objective and instructional objective are: 

a. Interchangeable . 

b. lower and higher order objectives, respectively. 

c. unrelated to each other. 

d. similar, but used differently. 



O 

ERIC 



107 



108 



Behavioral Objective 2 

Given sample objectives, the student will be able to identify 
the components which are missing, 

j 

Que s tions 

070216 "Given a list of 3 behavioral objectives, the student will be 
able to orally state which component(s) are missing, with 
100% accuracy. " 

The following is missing: 

a. important conditions c. criterion performance 

b. observable behavior d. nothing 

070217 "Given a list of 50 state capitals, the student will demonstrate 
that he knows the respective states with 100% accuracy." 

The following is missing: 



a. observable response c. criterion performance 

b. important conditions d. nothing 

070219 "W ith 90% accuracy, the. student will be able to arrange words 



in writing) in alphabetical 
letter of each word." 

The following is missing: 

a. observable response 

b. criterion performance 

070225 "The student will, by sight, 
letters of the alphabet." 

The following is missing: 

a. observable behavior 

b. important conditions 

070226 "The student will write the 
the United States with 100% 

The following is missing: 

a. observable response 

b, important conditions 



order according to the beginning 



c. important conditions 

d. nothing 

correctly identify all (26) 



c. criterion performance 

d. nothing. 

names of the last 10 Presidents of 
accuracy." 



c. criterion performance 
d* nothing 
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PRETEST FOR PROGRAMMED INSTRUCTION COURSE 



The following is a brief examination which will help us in determining 
your prior knowledge on some of the topics that will be taught in this 
course. Please work as quickly as possible, answering as many of the 
questions as you can. Do not be discouraged if you cannot answer many 
of the questions since all of the needed information will be presented 
throughout the term. 

1. Briefly explain what is meant by the use of the Systems Approach in 
the design of instructional materials. 

2. List, in order, the sever steps in Dick f s Systems Approach Model. 

3. List the three elements basic to all Systems Approach models. 

4. List at least three of the basic requirements necessary in choosing 
a subject matter area to be programmed. 

5. Briefly define: 

Frame - 

Cue - 

Response mode - 

6. Briefly differentiate between programmed instruction material and 
non-prcgrammed material. 

7. Define task analysis in reference to instruction design. 

8. List three characteristics of the learner that are considered as 
Entry Behaviors . 
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9. Briefly define; 

Discrimination Frame - 
Baboon Frame - 

. Constructed Response Frame - 

10. List the three necessary conditions of a well written behavioral 
objective . 

11. List Gagne’s eight conditions of learning. 

12. Briefly define; 

Intrinsic Programming - 
Extrinsic Programming - 
Adaptive Programming - 

13. List at least two reasons for selecting a particular programming 
strategy . 

14. List five characteristics of well-written test items. 

15. Name two types of formative evaluation. 

16. List the differences between using the Systems Approach in 
developing materials for programmed instruction and materials 
for any other medium. 
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MIDTERM EXAMINATION FOR PROGRAMMED INSTRUCTION COURSE 



This examination covers the thirteen cognitive units of the programmed 
instruction course. Read and answer all questions carefully. 

1. Briefly explain the major use of the Systems Approach in Education. 

2. List, in order., the steps in Dick's or Hansen's Systems Approach 
Mode 1* 

3. List the three elements basic to all Systems Approach Models. 

4. List five of the basic requirements necessary in choosing a subject 
matter area to be programmed. 

5. Briefly define: 

a. Step size-- 

b. Explicit response-- 

c. Feedback-- 
d* Prompt — 

6. Name three characteristics that differentiate programmed 
instruction material from non-prcgrammed material. 

7. Define task analysis in reference to instructional design. 

8. Briefly define what is meant by entry behavior (give examples). 

9. Briefly define: 

a. Baboon frame — 

b. Discrimination frame-- 

c. Confirmation frame-- 

d. Sub-terminal frame-- 

10. A well written behavioral objective contains: 
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11. List and briefly define Gagne's eigjit levels of learning. 

12. Briefly define: 

a. Adjunct programming* — 

b. Extrinsic programming- - 

c. Intrinsic programming — 

d. Linear programming — 

13. List at least two programming strategies and reasons for 
selecting each. 

14. List and define at least four types of test items. 

15. List five characteristics of a well-written test item. 

16. Briefly differentiate between formative and summative evaluation. 

17. List the differences between using the Systems Approach in 
developing materials for programmed instruction arid materials 
for any other medium. 
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PRODUCTIVE CRITERIA 
UNIT 7 



Productive Behavioral Objective 

The student is to write behavioral objectives for each unit 

according to the three required elements for behavioral 

objectives . 

Criteria 

1. Do you have a behavioral objective for each sub-task on 
the task analysis? 

2. Is the performance in observable student behavior terms 
(in action verbs)? 

3. Did you use words such as: know, comprehend, realize, 
understand? 

4. Do all behavioral objectives state the conditions under 
which the behavior is to occur? 

5. Do all behavioral objectives state performance- level? 

6. Are your behavioral objectives relevant? 



PASS: 

FAIL: 
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PROJECT EVALUATION SHEET 

Extra 

1. Task Analysis (10) Points 

2. Behavioral Objectives (10) 

a. Comparison with conventional curricula (1) 

b. Outline of contents of program (1) 

c. Show limits of the program T s objectives and 

area not being developed by the program (1) 

3. Description of Entry Behaviors (5) 

4. General Description of Target Population (5) 

(Not entry behaviors) 

a. Minimum grade on standardized ability or 

aptitude tests (1) 



5. Program Preparation (Description of preparation 
process) (10) 

a* Expert Opinion (1) 

b. Revision based on one-on-one (1) 

6. Evaluation Plan (Preparation for evaluation, data 
collect ior. and development of instruments) and 
report of data (error rate of program and test- 
ing) (10) 

a. Relationship of field test and target popula- 
tion (1) 



7. Revision Suggestions or Implications (10) 

(Based on Report of Data in #6) 

8. Miscellaneous (General overall impressions) (5) 

a. Indications of student attitude (1) 

b. Practicality of program, e.g*, supplemental 

materials, reusability, costs, etc. (1) 

c. . Administration, e.g., conditions necessary 

for success, procedures for introducing 
students to program, etc. (1) 



TOTAL 




Points 
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COMPUTER FLOW CHART SHOWING LOGIC OF CMI COURSE 



The components of the CMI course were logically oriented and 
are presented in the flow chart in Figure * The experimental com- 
ponents are also presented in order to illustrate their relation to 
the entire course sequence* 

Switch 22 was used as a means of sequencing the material* A 
one (1) indicated that the student was able to choose any unit in any 
sequence he chose* By setting switch 22 to zero (0) the student was 
sequenced from Unit 1, to Unit 2, and so on to Unit 20. 

Switch 23 was used as a means of providing evaluation criteria 
for the products to be written by the student. One (1) in switch 23 
indicated that the student would be presented the criterion questions 
via the computer, while a zero (0) indicated that the student would 
receive the evaluation from the instructor. These two switches were; 
used as a means of branching the four experimental groups used in the 
design of the study. 
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Figure 4. Computer Logic Flow Chart for 
CMI System. 
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PROGRAM PRESENTATION 
UNIT 7 



(The following information was presented to the student when 
he chose Unit 7. G: computer dialogue; S: student response.) 

C; Unit 7 

Identify behavioral objectives which 
have been written correctly. 

Write behavioral objectives for your 
area of instruction. 

Your BID for this unit is: 

S: 10 

C': Do you wish information on any of the 

following? If so, please type in the 
appropriate letter. 

a. EXPERTS--Students who have completed 
the unit you are currently working on 
who might be able to help you. 

b. MESSAGES--Information pertinent to 
class (lectures, reschedules, and 
so forth) . 

c. UNITS--A listing of units you have 
chosen to date. 

d. NO CHOICE--Branch to next frame. 

S: d 

C: You will now be signed off to go study 

for the next unit. When you feel you are 
ready to continue with the program, 
return to the computer and sign on, 
please. Check your task analysis to 
determine if you must: 

a. study for a test 

b. write material for your program 

c. do both a and b 
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(The student was then signed off to go study. When he returned 
to the terminal, he was presented the following.) 

G: Your name is: 

Joe Doe 

Have you signed -on to your own program? 
a. Yes b. No 

S: Type a or b. 

C: a 

You will now be given the test on 
Unit 7 

If this is incorrect, you chose and/or 
studied a different unit, type in the 
letter e, for error. 

If this is correct, space to continue 
with the test. 

S; (Spate) 

C; In order to evaluate the reference 

material which has been presented for 
you to study, it is necessary for you 
to answer the following questions. In 
refering to a reference, such as that 
of Mager, Preparing Instructional 
Objectives, you are to type in only the 
number of that reference as it appears 
in the Bibliography. The numbers for 
the references also appear before each 
cf the references in each unit. When 
you have typed in a number, space, then 
type in another, space, and so forth. 

Space to continue. 

S: (Space) 

G: Remember to type in only the number(s) 

and space after each number. 

1. Which references did you study for 
this unit? 
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S: 12 32 69 

0 

C: 2. Which references were of no value in 

studying for this unit? 

S: 12 

C: 3. Which references were of the most value in 

studying for this unit? 

S: 32 

C: The following statements were made by some 

students describing the way they felt while 
working on the section you have just completed 

Press the space bar to continue. 

S: (Space) 

C: I am tense 

1. not at all 2. somewhat 
3. moderately so 4. very much so 
Type the number to the left of the 
statement to indicate how you felt 
during the section of the course you 
have just finished. . 

Remember: indicate how you felt. 

S: 1 



C: 


I 


feel at ease 














i. 


not at all 




2. 


somewhat 








3. 


moderately 


so 


4. 


very much 


so 


S: 


2 














C: 


I 


am 


relaxed 














1. 


not at all 




2. 


somewhat 








3. 


moderately 


so 


4. 


very much 


so 


S: 


4 














C* 


I 


am 


jittery 














1. 


not at all 




2. 


s omewha t 








3. 


moderately 


so 


4. 


very much 


so 



S: 1 
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C: I feel calm 

1. not at all 2. somewhat 

3, moderately so 4. very much so 

S: 4 

C: The first question now follows. 

In a well stated behavioral objective, 
which of the following should be 
identified by name and defined in 
words of action? 

a. criterion test 

b. criterion performance 

c. observable response 

d. important conditions 

S : a 

C: c is the correct answer- Space, please. 



"Given a list of 50 state capitals, the 
student will demonstrate that he knows 
the respective states with 100% 
accuracy. " 

The following is missing: 

a, observable response 

b. important conditions 
c „ criterion performance 
d. nothing 

S : a 

The following statements were made by 
some students describing the way they 
felt while they were working on the 
section you have just completed- 

(The student was then presented the anxiety scale, as before, 
after which he received the following remedial information ■ ) 

C: You should study references for Behavioral 

Objective 1. Space to continue. 
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You must rebld on this unit since your errors 
were more than 20 percent. Type your bid. 

S: 3 

C: You will now be branched to CHOOSE- When you 

return to take the test, you will be tested 
on those objectives you missed. Space, please. 

S: (Space) 

C: Do you wish information on any of the following? 

Please type in the appropriate letter. 

a. EXPERTS --Students who have completed the 
unit you are currently working on who 
might be able to help you. 



(The student would then be signed off to restudy for Unit 7. 

When he passed all objectives for the unit, he would be given the 
criteria questions * for the productive behavioral objective as follows.) 

C: The following criteria are to be used in order for 

you to evaluate your written product for this unit. 

You must score it pass or fail. 

Criteria; 

. Do you have a behavioral objective for 
each sub-task on the task analysis? 

S ; yes 

C: 2. Is the performance in observable student 

behavior terms (in action verbs)? 

S : yes 

C: 3. Did you use words such as: know, comprehend, 

realize, understand? 

S : no 

C: 4. Do all behavioral objectives state the con- 

ditions under which the behavior is to occur? 



0 




S: yes 
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C: 5. Do all behavioral objectives state 

performance level? 



S: yes 

C: 6. Are your behavioral objectives relevant? 






S: yes 

C: Now you are to indicate whether you have 

passed or failed this product evaluation. 

a c Pass b. Fail 

Ss a 

Cs COMMENT FRAME. Type and enter each comment. 
When you have completed all comments, type 
$ and enter it. 



(The student could then enter any pertinent comments about the unit 
he had just finished.) 

S: $ 

C: This is where you choose the unit you will work 

on next. Type the letter associated with the 
unit of- your choice. 



a . 


Unit 


i 


k. 


Unit 


ii 


b. 


Unit 


2 


1 . 


Unit 


12 


c . 


Unit 


3 


m. 


Unit 


13 


d. 


Unit 


4 


n. 


Unit 


14 


e . 


Unit 


5 


0 . 


Unit 


15 


f . 


Unit 


6 


P- 


Unit 


16 


g- 


Unit 


7 


q- 


Unit 


17 


h. 


Unit 


8 


r . 


Unit 


18 


i . 


Unit 


9 


s . 


Unit 


19 


j • 


Unit 


10 


t . 


Unit 


20 



• S: h 

C: Please type in again. Check accuracy. 

S: h 

(The student was then branched to Unit 8, and the same general 
format was followed as was presented for Unit 7.) 
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GENERAL INFORMATION QUESTIONNAIRE 



Name o . 

Social Security Number ^ge 

^ajor Minor 

Highest Degreu_ 

Graduate Record Exam ~~ MATH ^ ___ ENGLISH 

Present GPA Taking Course for Credit Audit 

Phone 

Have you completed any of the following courses? (Check if yes) 

A . EDR 540 - Briggs 

B. EDR 660 or 552 - Esbensen 

C. Other courses relating to programmed instruction (Explain) 



Have you read any of the following books? (Check if yes) 



A. 

B. 

C. 

D. 

E. 
F- 

G. 

H. 

I. 

J. 

K. 

L. 

M. 



N. 



Instructional media - Briggs 
Sequencing of instruction - Briggs 

The Psychology of learning and instruction - DeCecco 
An introduction to programmed instruction - Deterline 
Working with individualized instruction - Esbensen 
Developing programmed instruction materials - Espich & 
Williams 

Teaching machines and programmed instruction - Fry 
The Conditions of learning - Gagn£ 

Teaching machines and programmed instruction II - Glaser 
A Guide to programmed instruction - Lysaught & Williams 
Preparing instructional objectives - Mager 
Good frames and bad - Markle 

Learning and programmed instruction - Taber, Glaser, & 
Schaefer 

Other related books (List below) 
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STUDENT ATTITUDE TOWARD COMPUTER MANAGED INSTRUCTION 
Developed by Bobby R. Brown 



This is not a test of information; therefore, there is no one "right" 
answer to a question. We are interested in your opinior on each of the 
statements below. Your opinions will be strictly confidential. Do not 
hesitate to put down exactly how you feel about each item. We are 
seeking information, not compliments; please be frank. 



Name: 



Date 



Name of Course Student Number 

CIRCLE THE RESPONSE THAT MOST NEARLY REPRESENTS YOUR REACTION TO EACH 
OF THE STATEMENTS BELOW: 



1. While taking Computer Managed Instruction I felt challenged to do 
my best work. 

Strongly Disagree Uncertain Agree Strongly 

Disagree Agree 

2. The material presented to me by Computer Managed Instruction caused 
me to feel that no one really cared whether I learned or not. 



Strongly Disagree Uncertain Agree Strongly 

Disagree Agree 

3. The method by which I was told whether I had given a right or 
wrong answer became monotonous. 

Strongly Disagree Uncertain Agree. Strongly 

Disagree Agree 



4. I was concerned that I might not be understanding the material. 

Strongly Disagree Uncertain Agree Strongly 

• Disagree Agree 



5, I was not concerned when I missed a question because no one was 
watching me anyway. 



Uncertain 




Strongly 

Disagree 



Disagree 
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Agree 



Strongly 

Agree 
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6 . 



While taking Computer Managed Instruction I 



felt isolated and alone. 



All the 
time 



Most of Some of Only Never 

the time the time occasionally 



7. 



While taking Computer Managed Instruction I felt. as if someone 
were engaged in conversation with me. 



All the 
time 



Most of 
the time 



Some of Only Never 

the time occasionally 



8. " The responses to my answers seemed appropriate. 

All the Most of Some of Only Never 

time the time the time occasionally 

9. I felt uncertain as to my performance in the programmed course 
relative to the performance of others. 



All the 
time 



Most of Some of Only Never 

the time the time occasionally 



10 . 



I found myself just trying to get through the material rather 
than trying to learn. 



All the 
time 



Most of Some of Only- Never 

the time the time occasionally 



11. I knew whether my' answers were correct or not before I was told. 

Quite often Often Occasionally Seldom Very 

•Seldom 

12. I guessed at the answers to questions. 

Quite often Often Occasionally Seldom Very 

Seldom 

13. In a situation where I am trying to learn something, it is 
important to me to know where I stand relative to others. 



Strongly Disagree . Uncertain Agree 

Disagree 



Strongly 

Agree 



14. I was encouraged by the responses given to my answers of questions. 



Strongly Disagree Uncertain Agree 

Disagree 



Strongly 

Agree 



O 

ERIC 



134 



15. As a result: of having studied some material by Computer Managed 

Instruction, I am interested in trying to find out more about the 
subject matter. 



Strongly 

Disagree 



Disagree 



Uncertain Agree Strongly 

Agree 



16. 



In view of the time allowed for learning, I felt too much material 
was presented. 



All the 
time 



Most of 
the time 



Some of Only Never 

the time occasionally 



17. I was more involved in running the machine than in understanding 
the material. 

All the Most of Some of Only Never 

time the time the time occasionally 

18. I felt I could work at my own pace with Computer Managed Instruction. 

Strongly Disagree Uncertain Agree Strongly 

Disagree Agree 



19. Computer Managed Instruction makes the learning too mechanical. 



20 . 



21 . 



Strongly 

Disagree 

I felt as if I 
Instruction. 

Strongly 

Disagree 

I was aware of 

Strongly 

Disagree 



Disagree Uncertain Agree Strongly 

Agree 

had a private tutor while on Computer Managed 



Disagree Uncertain Agree Strongly 

Agree 

efforts to suit the material specifically to me. 

Disagree Uncertain Agree Strongly 

Agree 



22. I found it difficult to concentrate on the course material 
because of the hardware. 

All the Most of Some of Only Never 

time the time the time occasionally 

23. The Computer Managed Instruction situation made me feel quite tense. 



Strongly 

Disagree 



Disagree 



Uncertain 



Agree 



Strongly 

Agree 
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24, Questions were asked which 1 felt were not relevant to the 
material presented. 

All the Most of Some of Only Never 

time the time the time occasionally 

25. Computer Managed Instruction is an inefficient use of the 
student's time. 



Strongly Disagree 

Disagree 



Uncertain Agree Strongly 

Agree 



26. I put answers in knowing they were wrong in order to get infor- 
mation from the machine. 



Quite often Often Occasionally Seldom Very 

Seldom 



27. Concerning the course material I took by Computer Managed 
Instruction, my feeling toward the material before I came to 
Computer Managed Instruction was: 

Very Favorable Indifferent Unfavorable Very 

favorable unfavorable 

28. Concerning the course material I took by Computer Managed 
Instruction, my feeling toward the material after I .came to 
Computer Managed Instruction is: 

Very Favorable Indifferent Unfavorable Very 

favorable unfavorable 



29. I was given answers but still did not understand the questions. 

Very often Often Occasionally Seldom Very 

seldom 

30.. While on Computer Managed Instruction I encountered 
mechanical malfunctions. 

Very often Often Occasionally Seldom Very 

Seldom 

31. Computer Managed Instruction made it possible for me to learn 
quickly . 

Strongly Disagree Uncertain Agree Strongly 

Disagree Agree 
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32. 1 felt frustrated by the Computer Managed Instruction situation. 

Strongly Disagree Uncertain Agree Strongly 

Disagree Agree 



33. The ^ ses to my answers seemed to take into account the diffi- 





culty o. the 


question . 










Strong i y 
Disagree 


Disagree 


Uncertain 


Agree 


Strongly 

Agree 


34. 


I could have 


learned more 


if I hadn't felt 


pushed . 






Strongly 

Disagree 


Disagree 


Uncertain 


Agree * 


Strongly 

Agree 


35. 


The Computer 


Managed Instruction approach 


is inflexible. 






Strongly 

Disagree 


Disagree 


Uncertain 


Agree 


Strongly 

Agree 



36. Even otherwise interesting material would be boring when presented 
by Computer Managed Instruction. 



Strongly Disagree Uncertain Agree Strongly 

Disagree Agree 

37. In view of the effort I put into it, I was satisfied with what I 
learned while taking Computer Managed Instruction. 



Strong 1 '?’ Disagree Uncertain Agree 

Disagree 



Strongly 

Agree 



38. In view of the amount I learned, I would say Computer Managed 
Instruction is superior to traditional instruction. 

Strongly Disagree Uncertain Agree Strongly 

Disagree Agree 

39. With a course such as I took by Computer Managed Instruction, I 
would prefer Computer Managed Instruction to traditional instruction 

Strongly Disagree Uncertain Agree Strongly 

Disagree Agree 



40. I am not in favor of Computer Managed Instruction because it is 
just another step coward de-personalized instruction. 

Strongly Disagree Uncertain Agree Strongly 

Disagree Agree 
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INFORMATION QUESTIONNAIRE 



Please be straight forward in your appraisal of your attitudes toward 
this class. The material will not be analysed prior to turning in 
grades. Read and answer every one of the questions. Please. Your 
responses will assist in the revision and evaluation of the course# 

Circle the number to the left of your choice, as: 

Do you like rain? 

(l) Yes, very much 

2 Sometimes 

3 Not at all 

When comments are required, be precise. 



1, What was the one most important feature of the course in terms of 
content ? 

2. What was the one least important feature of the course in terms 
of content? 



3. What was the one most important feature of the course in terms of 
procedures ? 

4. What was the one least important feature of the course in terms 
of procedures? 

5. Graph your attitude toward this course as the term progressed. 



VERY FAVORABLE 
INDIFFERENT 
VERY UNFAVORABLE 




6. What grade do you expect in this course? * 

7. What was your attitude toward your grade for this course at the 
beginning of the quarter? 



1 Very concerned 

2 Somewhat concerned 

3 Not at all concerned 

138 



139 



8. What is your attitude NOW toward grades for this course? 

1 Very concerned 

2 Somewhat concerned 

3 Not at all concerned 

9- Would you take another course by Computer Managed Instruction? 

1 Gladly 

2 Possibly 

3 Reluctantly 

4 Never 

10* Did you feel you learned what was expected of you as indicated by 
the behavioral objectives? Comment: 



11. Do you feel the material in this course was relevant to your 
professional goals? 

1 More relevant than other courses 

2 As relevant as other courses 

3 Less relevant 

4 Not at all relevant to my goals 

12. Would you have preferred n text and lecture classes? 11 

1 Absolutely, for this course 

2 Maybe 

3 Not at all for this course 

13. - Did you feel you had too much work to do in this course? 

1 Way too much 

2 More than most courses, but not too much 

3 Same as most courses 

4 Less than most courses 

5 Not enough work 

14. In what ways did you interact with the professor in the course? 
Circle as many as are appropriate. 

1 Source expert — he assisted in locating needed information 

2 Counseling and guidance — assisted in making decisions 

3 Group leader -- for small group discussions, lecture 

4 Evaluation expert -- merely scored papers 

5 Not at all 

15. Approximately how many times did you interact with the professor ? 
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16. In what ways did }'ou interact with the graduate assistants in 
the course? Circle as many as are appropriate. 

1 Source expert -- assisted in locating needed information 

2 Counseling and guidance -- assisted in making decisions 

3 Group leader -- for small group discussions, lecture 

4 Evaluation expert -- merely scored papers 

5 Not at all 

17. Approximately how many times did you interact with the assistants? 



18. Approximately how often did you ask fellow students for help 
or discuss problems? 



19. What was the result of 18? (Use back of page for comments.) 



20. Were you able to find the material needed to meet the objectives? 

1 All of the time 

2 Most of the time 

3 Sometimes 

4 Rarely 

5 Never 



21. .Do you think this class should have met: 

1 Every period designated for the class 

2 Every other period 

3 One hour a week 

4 Not at all 

5 Other 



22. Regardless of the source (computer or graduate assistant) , do you 
feel that the criterion questions for the products you wrote were: 

1 Extremely relevant 

2 Most were relevant 

3 Some were relevant 

4 Lacked some relevance 

5 Extremely irrelevant 

23. Also, were the criterion questions: 

1 Extremely adequate 

2 Most were adequate 

3 Some were adequate 

4 Lacked some adequacy 

5 Extremely inadequate 
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24. Do you feel that the product for UNIT 20 is the beist you could 
produce? 

1 Extremely confident that it is 

2 Quite confident 

3 Average 

4 Not very confident 

5 Not at all confident that it is 

25. Do you feel that the TESTS on the computer were a learning 
experience? 

1 Very much so 

2 More than most tests 

3 About average 

4 Less than most tests 

5 Not at all 

26. Which UNIT TESTS did you fail at least once? (Not those you had 
to take again due to machine errors . ) 

1 2 3 4 5 6 7 8 9 10 11 12 13 

27. Which UNITS did you fail due to poorly or improperly written, 

questions ? 

1 2 3 4 5 6 7 8 9 10 11 12 13 

28. Your final BID total according to your calculations:. . 

29. On UNIT 13, your calculations gave a BID total of: 

The computer gave you a BID total of: 

30. Comment on your reaction to BID during the first 13 Units. 

31. Describe how you paced yourself through the course (fit it into 

. the requirements of other courses, so many Units per week, etc.) 

32. When were you most prone to schedule time at the CAI terminal? 

Days: 

Hours: 

33. Ideally, how and when would you have liked to have had terminal 
time made available to you? 

How: 

When : 
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COMPUTER-EVALUATED GROUP ONLY 

34. Circle those units on which you fell the least competent when 
evaluating the product for the unit. 

2 4 5 7 8 10 11 14 15 16 17 18 19 

35. Do you feel that you were able to adequately evaluate your own 
products according to the criteria presented on the computer? 

Comment : 



36. Would you have wanted more interaction with the instructor of 
the course--more feedback? Comment: 



* * * * * 
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GENERAL QUESTIONNAIRE SHEET 



What area did you initially choose to program? 

Who was the expert (professor, etc.) you asked 
for assistance? 

What area did you finally program? 

(If same as above, write SAME.) 

Who was the expert (professor, etc.) you asked 
for assistance? 

(If same as above, write SAME.) 

Please comment on your relationship with the 
expert you chose. Favorable, unfavorable, 
helpful, and so forth. 



Use the remainder of this page to complain, praise, criticize, and cry. 
Do not fear, this page will be read after grades are turned in. We 
want an honest statement on anything you feel is pertinent to the class. 
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TIME RECORD 



Date 


Time: 


TOTAL 


UNIT 


CMI 

# 


STUDENT 

NAME 


SUMMARY OF HELP 


START 


END 


TIME 
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BIBLIOGRAPHY FOR PROGRAMMED INSTRUCTION COURSE 
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BID: A game of self-directed learning 

An instructional game used to give players experience in 
judging their own abilities and provide motivation for 
learning. Originally developed by Thorwald Esbensen during 
the implementation of individualized instruction in the 
Duluth, Minnesota, public schools. 

* 

Cognitive 

That part of instruction involved with the act or process 
of knowing; knowledge. Term used to distinguish the acqui- 
sition of knowledge from the utilization of that knowledge. 

Computer-Assisted Instruction (CAI) 

Refers to the use of a computer to present instructional 
materials to a student. Allows for immediate feedback, 
self-pacing, branching, and recordkeeping. 

Computer-Managed Instruction (CMI) 

Uses the capability of the computer to record the progress 
of a student through a learning sequence, providing guidance 
and control as dictated by the needs of the situation* 

CMI is further defined by the components in each specific 
system. 

Contract 

A student guide, containing descriptive unit title, 
behavioral objectives, and related references. 

\ 

Conventional instruction 
See Traditional 

Coursewriter II 

A programming language used to transform English statements 
into computer operable commands. 

Criterion 

A standard or established level of attainment which must 
be reached by the student. In the context of CMI, it 
usually refers to a test situation which is related directly 
to a course objective. (See criterion referenced testing.) 
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* 



Criterion Referenced Testing 

A testing situation in which a student is evaluated on how 
well he achieves the established criterion. 

Data print-out 

An organized collection of data produced for human in- 
spection and analysis by the computer. 



Data process ing* 

Any operation or set of operations performed on date which 
is generated in the student -terminal interaction. 



Debug* 

To test for, locate and correct errors. 



Encode* 

The process of transforming commands according to a specific 
code, suitable for input to the computer. 

Feedback 

Presentation of information to the student based on an 
analysis of his responses. This may be delayed or 
immediate . 

Flow Chart* 

A graphic representation for the solution of a problem; 
graphic display of the logic desired for a computer program 
presentation. 

Individually Prescribed Instruction (IPI) 

Instruction based on prior and initial capabilities of a 
student, prescribed according to the teacher's estimation 
of correct alternatives which match a student's profile. 

Individualized Instruction 

Adapting instruction to individual requirements, providing 
for self-directed learning, self-pacing, independent study, 
and one-to-one teaching when required. 



Logic* 

To condense representations and avoid the ambiguity of 
natural languages. The decision structure for computer 
processing of information. 



*Adapted from Jordain, P. B. Condensed computer encyclopedia . 
New York: McGraw-Hill Book Company, 1969. 
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Productive (Product) 

That part of instructional sequences which deals with the 
use of prior learned knowledge in the writing or development 
of written materials. 

Programmed Instruction (PI) 

Narrowly defined as a method of presenting learning materials 
in frames which require responses from the student. Broadly 
defined as a method of presenting the instruction in an 
organized , systematic manner, with feedback to the student. 

Segment* 

A hardware defined portion of a storage area (track) having 
fixed data capacity. 

Student -Terminal interaction 

The use of a computer in which the student utilized a terminal 
for on-line dialogue with the computer. 



Switch* 

A hardware device for indicating that one of several alternative 
states or conditions have been chosen. 

Systems Approach to the Development of Instructional Materials 
A method, plan, or pattern, to use in order to prepare 
instructional materials. Most systems approach models 
include, as a minumum set of requirements, the establishment 
of objectives, presentation of instruction, evaluation, 
and feedback. 

Task Analysis 

A process for determining the subordinate learning tasks 
required for a given sequence of instruction. Assumes that 
a subordinate task must be completed prior to the learning 
of any superordinate task. 

Technology 

Materials which can be used to aid man's work; computer, 
globe, blackboard, pencil and paper, and so forth. Also 
sometimes used to refer to the systems approach for the 
design and utilization of instructional materials. 

Terminal* 

A device by which data can leave or enter a computer system; 
as used in CMI, a cathode-ray-tube display (similar to a TV 
screen) with alphanumeric typewriter keyboard. 
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Traditional 

The type of situation encountered ,in schools which are teaching 
by the lecture method, from prepared texts, with one teacher 
per contained classroom. 

User's file 

The status of a student's performance record at the last 
sign off. A file automatically -maintained by the computer 
in order to determine "where students are" in a set of 
computer presented materials, whether instructional or not. 
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